Past the Productivity Ceiling: Rebuilding the Enterprise from First Principles

23:00Matt Paige:It is funny. There's like this model photo that exists. It's Hey, maybe I should go down to sonnet, but what if I miss out on something using Opus? I like, I never do it. You hit the limit and you're like, okay, let me sit around here for a minute or this morning. I don't know if you got hit, but Claude was just down for a few minutes and it's like, okay. What is like the end of the world? Right? Exactly. Exactly. But you hit on something you hit on GitLab, the agent platform. I'd love to like hear more about, and obviously this is on the product side. I'm assuming you being on the internal side and focusing there, you're probably leveraging GitLab to build GitLab and things like that internally.

23:37Matt Paige:But how do you think about that and preface it with, it's this unique moment where anybody can now build things with AI. And I've built probably over 100 different applications and things. There's probably like three that are actually out there in the market. And I think that's with a lot of people, right? And it's that kind of last mile, but like, talk to me, like you've had this GitLab duo and how you think about agentic orchestration and whatnot across the SDLC. Yeah. I'll talk about it in two ways. I think one, how we leverage it internally, but also why I think it matters more broadly for the industry.

24:16Certainly we see there are a lot more builders out there, right? And that's an incredible thing to see. The barrier when it comes to creating an application is on one hand down incredibly small. But as we think about all the other challenges of how you utilize something, and it's again, things like security, like how and where do you host it and build it? And just like all of those more complex challenges. I think that's how we leverage dual agent platform internally is it's our ability to really give team members that opportunity to build that scale, but still by enforcing the level of governance in control that we need to, both as a public institution, as well as one that has a lot of regulated customers.

24:54And I think that is a distinction that is easy to forget about as we talk more broadly about the advances in AI and AI for coding. There's a lot that happens client side today, whether using Codex or Cloud or even open source models. And you can do so much as sitting on your local computer, right? But I like to think about this distinction between local versus repo side. And when you think about what's happening repo side or what's happening server side, that's ultimately where you can put in assurance and governance and guardrails and the things that make you really comfortable about what's being developed.

25:29And I think that's the balance that we find with dual agent platform is we say develop software in whatever mode and approach you want, but ensure that you have the right level of governance on that repo side. But leveraging AI agents to make that as increasingly low lift as possible. And we think about code reviews as a really common use case. We have AI-generated code reviews. You can do auto-remediation for security vulnerabilities that are found. You can do as a pipeline is running to actually push the code out. If that has a failure, we can use AI for auto-remediation on those pipelines. And so we find we can really compress the after-code part of the software lifecycle really heavily using the Duo Agent platform.

26:10But that's something you can't natively or easily do if you're just running AI locally.

26:16Matt Paige:Yeah. Yeah. It's interesting. There's this nuance to, I've noticed the concept of human in the loop, which I feel like was talked about a lot over the past couple of years. And like, you have to have the human in the loop, but like what AI was capable of doing, the humans physically can't keep up. It'd be in the loop. So it's almost as the bottleneck continues to shift. To your point, we're leveraging AI in areas where we maybe used to have human in the loop, but there's still human in the loop, but it's at different levels of abstraction maybe i don't know any thoughts on that nuance and how you think about human in the loop and then because i feel like the models are getting so good that like you can't really leverage in this kind of check and balance mechanism that you're mentioning yeah i think you kind of hit the nail on the head where the nature of the human in the loop is evolving right and so it's i think if we went back even just a year ago the human in the loop was this hard gate that you had to pass where maybe they were even reviewing every line of code that was AI generated, right?

27:19And that's clearly not where we are today. As you think about agentic software development, you have teams that are creating so much code that there's no way a human could review every line that's being created, right? And so I think that's where increasingly we see people talking now about looping and loops and what does that mean? And it's agents working with other agents that have been an oversight agent, right? But there's still ultimately a person there that's overseeing that orchestration, overseeing the outputs, applying their own taste and judgment to what's being developed ultimately.

27:51And there will be this increasing abstraction of where the person gets involved. And I think that's just going to be the nature of the pace of development, the amount of code being generated. But I think that's also where it's really important that if you're in an organization, you understand what are those key points where you want to have a person involved and maybe you go look the code reviews we just want to read an ai output of code reviews but that's fine but actually the security scans we're always going to want a person to review right i think that's very much an organizational decision that it's going to become part of that evolving view around risk factors versus productivity but it'll take us a bit to catch up there it took us with traditional integrations and automations as a similar view 15 years ago where it's oh you can't automate things directly and you need to have a human reviewing every transaction, approving it.

28:38And now we take that for granted where we clearly have deeply integrated systems that are currently using deterministic based integrations. The next iteration of that will be how do we drive assurance and governance across non-deterministic ones using AI. And I think you'll have governance type platforms like GitLab for software development that help drive that.

28:57Matt Paige:Yeah. It's one of the biggest things when we're talking to folks and clients is the governance side of it. It's just becoming the hot topic right now. But like the one book I'm like recommending most to people is called thinking in systems. And I think your ability to understand how systems work is just super important because it's understanding at that level of abstraction or first principles in terms of systems and feedback loops. You mentioned loops earlier and self-reinforcing and balancing loops. And the example everybody uses is like the thermostat where it's always keeping things back in check and whatnot.

29:33Matt Paige:But you mentioned the term loops and this is like the new hot thing with agents and whatnot and the concept of a loop is not a new concept but it is the thing that a lot of people are talking about can you go a bit deeper in terms of maybe what loops are and how to think about those from an ai and agentic standpoint yeah i think look i'll be honest like i personally am early on loop still i think it's something that as the just AI landscape evolves, right? It's tough to keep up with everything that's happening, but how we see loops both on our product side, as well as some of our developers are using it is really around a change in how you interact with AI.

30:17And I think this is part of that evolution we've seen where again, maybe a year and a half ago, it was just like chat prompts and code completion. And then you have a gentic coding where you're like, no, you're giving a bit more of a complete prompt and getting full code blocks back. But that evolution now is really to have a bit of a more slightly autonomous way of developing your software, but by having a discrete set of agents that are working on discrete sets of problems that reinforce one another. Right. And then the person is there that can still inspect outputs that can give corrective actions that can provide additional inputs into that.

30:53But ultimately, you have a set of individual agents that are working in their own loops. They're taking information from the other agents and ultimately self-correcting. And this is so interesting because in many ways, it's a different way of thinking about how you achieve your job now. It's not just about me, the person sitting and writing code and ultimately pushing an application. Or if I'm a sales rep, it's not about me looking at a single account, doing the research and saying outbound email, right? You're really looking at how a team of agents is working and operating and you're helping orchestrate them.

31:27And so it's a bit of the role of almost like a manager of agents as opposed to the doer of a task. And I think that's a bit of the evolution that we see really happening in the workforce.

31:35Matt Paige:Yeah, no, it's super interesting. And I think too, it's even when you're leveraging, whether you're working in chat, GBT and kind of the reasoning side, cloud code, code code, you can see a lot of the times that reasoning with itself, it's like, oh, I can't do this. Let me go do that. Oh, I researched that. That's super interesting. So you can almost witness this reinforcing loop in a sense where it's gathering context, doing the thing before it gives you the. Yeah. It'll be like, oh no, I can't use that tool. Let me try a different tool. Let me try this thing. Yeah. It's funny kind of watching there.

32:06Matt Paige:It's like a train of thought, which is super interesting. I mentioned earlier the AI accountability reports that just got published today or yesterday. And, you know, by the time this goes live, it will already be out there. But it was like feverishly giving that claw to go research it. But you surveyed 1500 DevSecOps professionals. And I don't know if you've actually seen the output yet because it is so current. but it argued that context and traceability are the new differentiators and that speed is basically getting commoditized. How do you think about that concept of context and traceability and why it is so important right now?

32:47Yeah, you and I spoke about it a bit earlier as well, right? If you have enough of the right context, AI can do a bit of anything. And I think that's been the case for a while and it's certainly still true today, but it's almost more true because as you think about the capabilities of the different frontier LLMs has increased and improved so dramatically, the ability to access context itself has not. And so I think that as you look at how do you bring the right level of context to a task, whether it's through automated means, a human introducing that context, it's increasingly important. And I'll use an example.

33:22First, it's from a team I was talking to last night, and then I'll use from a GitLab product perspective. But if you take a customer support workload, right? So we have a support engineer and GitLab is a fairly technical product. We have very technical support engineering team members. But on one hand, people say AI is definitely like customer support to solve a problem, right? But the problem is context still really matters. And how do you actually bring to bear all of the experience that an individual has when it comes to different deployment types, different operating systems, different versions of things, the problems that they've seen in the past.

33:59Very rarely is all of that documented because that's part of what a human brings to bear. And as we think about AI and customer support, for example, it's where there's rich context, we're going to leverage AI. But then there's areas where there isn't rich context that's documented and we're going to leverage the power of the person. As you think about in a software development workflow, the same thing's actually really true. So on one hand, you can have a code base, right? and the AI is great at working through a code base. But what about the failures when you tried to push that code? What about the last security scan?

34:30What about the last comments from your manager on the code review? And all of those things stitched together actually create this more relevant and rich context that can ultimately make your agentic operations a lot better. And we see as people are leveraging, for example, our own knowledge graph, GitLab Orbit, that you can consume less tokens, get more accurate output, and oftentimes quicker output because of the ability to have this really rich knowledge graph of context, as opposed to piecemealing together pieces of that.

34:59Matt Paige:Yeah, it's interesting. On my own personal side, I've been playing around with Andres Carpathi's LLM wiki that he created. And I don't know if you qualify that as a knowledge graph or it's not really RAG either, but it's just interesting. I feel like it's close enough though, right on a personal basis. Yeah, conceptually, right? But it's really cool how it does make these connections between the things based on whatever whatever I'm doing and whatnot. The other, there's two, two other stats on the report. I'm just curious to get your take on. So there's one that the headline is AI coding is paying off and 60 % of DevSecOps professionals, it's easy to say DevSecOps, say the ROI from AI coding is exceeding expectations, right?

35:42Matt Paige:And then right after that, it was interesting. It said, but 73 % of DevSecOps professionals are concerned about the long-term maintainability of AI generating the code. So on one side, they're like, oh, this is exceeding expectations, but we're also concerned about maintaining it. Do you think the concern is maintaining it from a quality standpoint or just maintaining it from, I don't know what the heck this thing is doing in a sense? Yeah, I think it speaks a little bit to the fear of just what's not known, right? And so it's a bit of, there's so much code being generated. our velocities increased and the traditional ways that we've managed that can't hold up just again like what we were speaking about earlier and so i think a lot of that is just fear of hey like we actually don't necessarily know what will come next and afterwards but i think that's a lot of why the governance side is really critical how can you enforce organization coding standards how can you ensure that code is getting reviewed and scanned and all those things because all that ultimately really helps drive down that fear of the unknown in the future.

36:50And it also helps speed up everything that comes after the development of code. One of the other things that we see increasingly is that while coding is a lot faster, that the bottlenecks actually moved. And I think in the report, it talks about how 85 % of people say that AI has moved the bottleneck from writing code to now reviewing and validating, right? And so I think like all of those things go hand in hand, that if you don't have a good strategy to manage that influx of code after it's written, that's a bit of where that fear of what happens next comes from.

37:22Matt Paige:Yeah, that's interesting. And I'd be curious to get your take on this too. I was thinking through this the other day, and this may not apply as much to GitLab, but there's that whole narrative around the SaaS apocalypse. And part of me is it's just completely overblown and there will be incumbents that don't make it through and new startups disrupt them, just a tale as old as time. But it's interesting. It's every tool and platform has like their own agents. But then from a user perspective, it's where I really want to use 30 different agents. And it's like this bring your own agent concept where, hey, I just want to like connect to your thing and let my agent or quad go do its work with my context for my other stuff.

38:03Matt Paige:But then I started thinking through it. But there is value in whatever tool or SaaS product you're using and it having an agent, but its core function isn't necessarily interacting with the user, but the user's agents. So we can get the most value out of the platform. I don't know, like, just curious, thinking on that, maybe from a broader landscape and agents and all the things going on. Yeah, look, I for one also do believe that the SaaSpocalypse is probably overstated. The nature of SaaS and how we use it is going to shift and change. But I think what doesn't is the need for key systems of record, the need for systems that apply governance and compliance and SOX controls and just all of those things that come with it.

38:45And look, like even from embarking on some of our own internal journeys, it's like really easy to get to that proof of concept that's 80%, 90 % of the way there. It has bells and whistles and features that people love. But that remaining 20%, as you start to layer in like role-based access control, as you start to think about things like versioning, as you start to think about how do you maintain immutable versus non-immutable records, like the things that are actually like really deep problems, those end up being this really long tail on internal development then. I don't think that AI-driven software development is going to fundamentally shift and change that, especially when we think about the types of workflows and workloads that run on top of some of our key SaaS offerings.

39:28I think your note around the agent sprawl and proliferation is super real. And every agent also then wants context to every other system of record and application, right? So you start to think about all of the kind of challenges around data sovereignty, data governance. It's a very real challenging problem. For us internally, our strategy has been to focus on a few, a handful of a few platforms where we really build out agents, where we allow our data to reside, but then leverage things like native MCP connectors or custom built MCP connectors to our SaaS applications so we can bring in that rich context.

40:04I think we'll see that evolve. I'm not sure if MCP is the be all end all protocol for what we ultimately all need to do. You're coming mad around, hey, are we going to have like agents talking to agents? And that actually is like really interesting. I think something that we'll probably see a bit more of like agent-to-agent communication protocols emerging. But I don't think that MCP itself is going to be where we all land in a few years.

40:25Matt Paige:Yeah, no, that's a good point. I think you hit the nail on the head. It's cool that you can go vibe code your CRM, but by the way, you have to maintain it, govern it, add features to it. And is that actually differentiating to what you're doing as a business or could you spend your time better elsewhere? And it's just, it's trade-offs. back to like first principles in a sense, right? But I'd be remiss if I didn't ask about shadow AI to a CIO, like shadow IT is like this concept that a lot of people know and are familiar with and maybe are culprits of, but how do you think about this concept of shadow AI and people using their own AI that maybe is not sanctioned or approved?

41:06It's just like with shadow IT, it's a problem that exists and it happens. you have both awareness on it and you're constantly combating it. I think a few things that maybe jump to mind, how we tackle shadow AI isn't really any different than how we've tackled shadow IT in the past, right? And it's a mix of education, policy, a level of controls that exist with a network and endpoint layer, as well as then, and I think the most critical, is having a really happy path to production for people, right? The official means of getting something approved, of being able to do a POC, of being able to deploy a vibe-coded app, those can't be more painful than going on your own.

41:48And so a lot of it's this deep interaction we have with our team member base to ensure that we're joined with them, we're in the journey with them, and we're here to support them. And if they have needs, ideas, maybe we'll try to steer it towards a different approach or platform that aligns with our governance strategy. But ultimately, to really have that happy path to production. I think we've struck a good balance internally around giving a fair amount of freedom in those individual sandboxes I talked about earlier, but then having some governance and control as we think about broader access to data and enterprise-wide deployments.

42:22And that's helped stem that. But look, I would say the biggest challenge is probably the Vibe-coded apps and how we manage that. But luckily, we're able to do so on our platform and drive a fair amount of governance through that.

42:35Matt Paige:Yeah, no, I think you hit it. It can't be more painful than doing it on your own. It's like the perfect barometer. Okay. Let's do a bit of a lightning round. I'm going to ask a few questions to wrap us up, but okay. So what superpowers do you think are most important for individuals to have right now in this new era?

42:56Matt Paige:Gosh. Or quality superpowers. Yeah. Look, I think intellectual curiosity has always been something I've said, and I'll say it even more with AI, your ability to access information to become a pseudo expert on things is unparalleled to any time in history before. And so if you have curiosity and a little bit of intellectual horsepower, you can pretty much do anything now. Yeah. Okay. So for the senior listening who suspects they're stuck optimizing old workflows instead of rebuilding them, thinking from first principles, what's the first move that they should make. If somebody's early on in this journey, what do you feel is the first move that they should do?

43:35I think it's going to sound a little trite, but it's executive team alignment around the scope of the change you want to make. And that was one of the first things I did was really align with my peers on the executive team that we're not looking for incremental gains. We're looking for a wholesale transformation. And that's ended up being a really tight partnership with our chief people officer in his organization as we think about things like the evolving nature of roles and AI literacy and enablement and all those things. But like that top level alignment on what are we even trying to do and driving that scope of change, I think is the most critical.

44:11Matt Paige:What's one AI capability you wish vendors would stop overselling? Or what do you think is overhyped right now? Yeah, look, I still think a lot of AI vendors are simply selling essential, essentially wrappers on a model. And so it's like, we're going to give access to the data in this SaaS app to some frontier model. And they're trying to upcharge you a lot for that. I think there's nominal value in that. We really have moved beyond just data-based type use cases with AI. And we really need to have true Agenta capabilities to get work done. What's the single best dollar of AI spend you made in this past year?

44:48I normally would say GitLab, but I don't have to pay for GitLab. I think one of the ones that's given us the best bang for a buck is a platform called Glean. It's essentially an internal enterprise search platform. And I think it both is an agent platform for us, but also knowledge platform. And Pretty University is one of the ones we get the highest pieces of feedback around. Yeah, we're actually partners of Glean.

45:10Matt Paige:It's that whole intelligence layer concept is just very top of mind. Yeah, it's kind of like you're talking about Carpathy's Knowledge LLM. This is the enterprise Knowledge LLM, right? Exactly, exactly. All right, last one for you. The advice, what advice would you give to your pre-AI self or maybe before you joined GitLab? What would you have told your younger self then that you know now? Yeah, I feel like the AI era is like living in dog years, right? But I think, and I've done an okay job with this, but I think I just remind myself, we need to be nimble. We need to be agile. We need to move quickly.

45:41And it's okay to pivot. And if I think about the nine, 10 months I've been at GitLab, we've probably had three different iterations of our AI program. And I think that's perfectly okay. What doesn't work is the traditional way of RFPs and POCs and all that stuff that like enterprise technology teams have traditionally done. We really need to have a more agile approach and be okay that you're going to make a call and maybe next year it's not going to be the right call anymore.

46:07Matt Paige:Yeah, it is interesting. The cost of pivoting, I feel like has gone down a lot, right? Because you can do that more. And I struggle with this at times too. I'm like, oh, we can't make that pivot again or change this, but it is easier to do it. So there's that balance between you're constantly optimizing toward whatever metric you want and then creating confusion. But there is value in it. I do agree with you there. And I think it's like in this current time period, speed is probably the more important metric than some of the traditional ones we've optimized for. And so I think when we get bogged down and again, traditional ways of thinking about problem solving, that ultimately hurts us more than licking your wounds if you've made a wrong call, but actually moving more quickly.

46:55Totally.

46:56Matt Paige:Matt, it was great having you on Talking AI. I really enjoyed the conversation. Where can folks find you, learn more about GitLab? Yeah. First off, Matt, thanks for having me. It's been a great conversation. I'd say I'm like pretty active on LinkedIn, do a lot of posts and invite people to connect with me. happy to always do coffee chats and connect. And then our GitLab blog is great to both learn about agentic orchestration and how we see the future of software developments. And really would love for people to check that out as well. Yeah, there definitely is some good stuff on there. All right, Manu, great having you on Talking AI.

47:28Matt Paige:Yeah, thanks so much, Matt. Thanks for listening to the Talking AI Podcast. If you enjoyed the show, give us a follow or subscribe on your favorite podcast platform. And don't forget to leave us a review. We love those. For more info on Talking AI, visit TalkingAiPodcast.com. Quick break in the pod. If you're listening to this podcast, chances are you've been thinking about how to actually use AI inside your business. And that's exactly why we built the AI Opportunity Finder. It's a free tool that helps you uncover high impact, tailored AI use cases based on your business, your goals, your pain points, and your industry.

48:04Matt Paige:No fluff, no generic use cases, just real ideas that fit your business and the rank by ROI potential. It takes about three minutes to run, and it's like having your own personal AI strategist for free. If you want to try it for free, check out the link in the show notes or go to hatchworks.com backslash AI dash opportunity dash finder.