How to Design an AI-Native Engineering Organization

0:00Today on the AI Daily Brief, how to design an AI native engineering organization. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:18Hello, friends. Quick announcements before we dive in. First of all, thanks as always to today's sponsors, Superintelligent, Plum, Vanta, and Agency.org. To get an ad-free version of the show, go to patreon.com slash AI Daily Brief. And today, as I am traveling, we have a different type of episode. I'm joined today by serial entrepreneur Brian Elliott and Sid Pardeshi, an ex-NVIDIA software architect with 27 generative AI patents to his name. Sid and Brian are the co-founders of Blitzy.com, which of course you have heard me talk about as a sponsor. Now to be clear, Blitzy's sponsorship does not include a sponsored episode.

0:56This is something that came out of a bunch of conversations that I had had with Brian and Sid offline that I thought would be interesting to share. Coding is obviously at this point one of, if not the most transformative use case of AI, particularly when you think about in the enterprise. It's also a use case that opens up more use cases and compounds the rate at which everything is changing. And so I think even for folks who are not building or in engineering organizations, understanding what's happening in and around AI and agendic coding is incredibly important. In this show, we talk about the barriers holding AI coding back in the enterprise, specific types of investments enterprises can be making to make agentic coding work better, and ultimately try to get into a blueprint for the modern AI native engineering organization.

1:38It's a fun conversation, so let's dive in. Hello, sirs. Welcome to the AI Daily Brief. How's it going? NLW, let's get into it. Glad to be here. Yeah, no, it's great to have you here. We're talking today about, I think, a topic that is, you got to say at this point, that if there is one definitive, like just no denying it use case for AI, specifically when it comes to sort of business and business operations, it's coding, right? Like everything else, there's tons and tons of things that are changing productivity and changing how people do their jobs. When it comes to like really rethinking fundamental structures, things that you could get fired for not doing, you know, re-imagining sort of the engineering function with the new opportunity of AI and agents is pretty high on that list.

2:24And so we're going to dig into that a little bit today. And where I want to start is almost more broad, right? Like for people who are not paying attention to sort of every shift and change when it comes to AI and coding and, you know, agentic coding, what is the landscape right now? What is it being used for? Is it primarily just sort of on the startup or individual side? What is the state of models? Kind of like, take us into the picture of AI and agentic coding, broadly speaking, in mid-25. Yeah, let me give a band of the simplest use case and the most complex use case. And this is being adopted across startups and enterprise.

3:03So from a simple use case perspective, you have GitHub Copilot, you have Cursor, you have these tools where you can give them a little bit of intent. Say, I want to create a function that creates a login screen. And that's going to give you back some code, a few hundred lines of code. And that's really useful because developers don't remember a specific syntax for a language or they want this to work with their specific brand guidelines. If there's something on the front end, you can provide that to the copilot. So that engineer within their IDE is going to be 20, 30 % more efficient. And for somebody that costs$100,$200,$300,$400 ,000 a year, depending on where you're based and how skilled you are, that is meaningful when extrapolated across the enterprise.

3:44So that's where we're started on the simple use case. Now, if you go to the other end of the spectrum, you can think about how long is that agentic system thinking and validating before providing the code back. So on the first part of the system where we had just the co-pilot, it's immediate response with the IDE, getting you something that might work or make you go faster versus thinking something like deep research, which we're all familiar with from a sort of research and report generation perspective, where you are able to agentically improve the quality of the code. And these are these much more agentic solutions that you're thinking about where you have tools like Devon or Factory that are going to take 30 minutes to come back and get you a chunk of work that might be hundreds or thousands of lines of code that is taking into account all of the relationships around that.

4:34And then you have us Blitzy, which is going to take 12 hours or a few days, or we've had multi-week runs, right, where you're going to be accounting for tens of millions, 20 million, 30 million lines of code and delivering huge chunks of work, right? And so you really have all the way from a little bit of intent, a little bit of code to huge amounts of inference time compute and huge amounts of code, ultimately delivering productivity, whether it's 30 % or 300 % really depends on the approach the enterprise is taking. Yeah, and if you dig deeper into Brian's response, right, if we focused on the modus operandi or the use of the tool, there's also something interesting to look at from a use case standpoint, right?

5:12So if you're looking at prototyping or building something that you can quickly demonstrate how it looks like in the UI, if you're a product manager at an enterprise, then it's very easy to use something like Figma Make or Google Stitch and get something out there quick that you can share a URL and have someone to see it. if you're doing something a little bit more complicated, like maybe a GitHub issue or a Jira ticket, that's a full feature or a bug, right? Then probably it makes sense to use something like an IDE tool. And even Blitzy plays into that space. But the point there is like you're writing hundreds, if not thousands of lines of code, right?

5:44But what if you're doing large scale modernizations, refactors? That's where Blitzy and some of these other tools can help you where you need very large-scale context, right? Like particularly for Blitzy, it's like we've pioneered infinite effective code context. I'm sure we'll talk about that later. But it's about keeping context across multiple repositories, modules, systems, and producing something that the humans can then review and take to production eventually. So, okay, so here's where I want to go with this because I think it's interesting. One of the things that has been surprising over call it the last six months is how frequently it is the engineering departments of organizations that we're interacting with, you know, with agent readiness audits and things like that, that are blockers to work.

6:31And sometimes it feels for very kind of, you know, personal reasons, like they chose a very cushy position for a specific reason and don't really want to work all that much harder. But let's hold that aside. There have been a set of kind of complaints or just observations about how not enterprise ready certain tools are. But it seems like that's just based on your answer, that's sort of changing fast. What has held back enterprise adoption of these processes on whatever end of that spectrum it is? I feel very strongly for this personally. So I think we talk a lot about it from a technological standpoint, but also there's the mindset standpoint that you spoke about, the change management.

7:10If you think about the approach that the AI tools are taking today, it's like, let's work with the human, right? Let's take some instructions from the human, get something back, review it, and push it. And that's kind of like the easy way to do it because essentially you're like building like a wrapper around Claude or Gemini or some of the other models. And then you're building something that's a quick response. And that is a response to the enterprise where the enterprise doesn't trust the models, right? You have LLMs that can produce code, but that code necessarily isn't the most high quality.

7:41You still have to spend time validating it, fixing it, and making sure it really adheres to your enterprise's best practices and whatnot. The challenge really for moving forward to the next step is how do we get enterprises to trust the AI, right? As of today, AI is probably the bottleneck. It's just not good enough for the humans to blindly trust it. But given the advances that we're seeing with new model releases like every couple of months between like Cloud 3.7 and Cloud 4, what's going to quickly happen is that humans will become the bottleneck. And the enterprises that are not quick at adopting the AI tools today are going to be significantly lacking at that point because they were slow to get the mindset change.

8:22In terms of technological adoption, we're not seeing like a huge lead time in adopting an AI tool. If you really clear all of the hurdles and from the security background checks and all of that, once you do that, like integrating is like literally a matter of days, right? But the hurdle or the biggest challenge for enterprise AI adoption is really getting your most senior architects to change their mindsets and perspectives on AI, on changing how you build code. Like, for example, rather than writing all of the code yourself, letting the AI write it and finish it off, right? Handing it a set of requirements which forces you to over-communicate.

8:59And engineers historically haven't been the best communicators, right? So how do you cross that hurdle, change that mindset, incorporate AI into your entire software life cycle, right? And make sure you're ready for the next wave is where we are seeing the biggest challenge. It's very hard for the enterprise if you look at it from their perspective because there are thousands of tools out there. And most of these tools are built by very, very young entrepreneurs that haven't spent time at senior levels within these enterprises. So they don't know what is important to them from an adoption perspective, right?

9:32And so they're building SaaS without the ability to deploy it within the client's VPC, which if you're going to work with a financial services institution, forget about it. They're not going to adopt SaaS. And so they're building these solutions without the enterprise in mind that have good value, but they're not designed to be procured in a way that the enterprise has to de-risk. And so you need both the technological advancements to do something really, really compelling, but you have to make technical design decisions up front that is set up for the enterprise to adopt how they can procure.

10:05Yeah, I mean, it feels like this is one of the... So in our conversations with enterprises who are bringing up these types of issues, it's a very clear skate to where the puck is headed moment when, you know, because it's just the obvious opportunity for companies in this space to sort of rebuild and redesign enterprise engineering efforts is so huge that there's a flood of companies that are now taking on different pieces of that. It's sort of one of those convenient short-term excuses that, you know, companies haven't been set up for that because that's not going to be the case for very long.

10:37One of the other things that I wanted to ask you about, which I think gets a little bit into your architecture and how you guys approach some of these issues, is, broadly speaking, we're sort of in the midst of an inflection between the assistant era and the agentic era. This involves kind of a reimagining of AI from AI that helps me do stuff to AI that does stuff for me. That all in of itself is a pretty big mindset shift, even for people who have spent the last couple of years getting comfortable with assistant type tools. However, I think that it still feels that even the folks in general who are kind of fully embracing and thinking about agents as collaborators and digital employees and even starting to imagine, you know, teams of agents working together are kind of under imagining the full capabilities of when you have sort of intelligence too cheap to meter that you can use, you know, at mass scale.

11:31Like, right, like we're still kind of in this mindset of replacing my copywriting with an agent rather than imagining, you know, 100 copywriters who are all competing, you know, in this sort of, you know, this war game scenario that I've set up. How are you seeing that type of shift happen? And maybe this is a chance to kind of talk about the swarms that are a part of what you guys do. Yeah. In general, I think people are dramatically underreacting to what is happening in the ecosystem right now. The rate of change is like nothing we've seen in our lifetime. And the ability to pivot an organization to be able to be set up to give work off to a system and then get work back is fundamentally very, very different.

12:14You have 20, 30, 40 ,000 employees. You're trying to re-architect the entire system that's built on people changing off work from one person to the other person to the other person. So we've been thinking about how to solve this problem of giving work off to a swarm of agents for a very, very long time now. And this process is fundamentally superior to the human bottleneck, human in the loop. For us, we have human in the loop at very specific checkpoints. But at the point where the human is sending that work off, letting the system get as high a quality of outcome as possible for whatever it costs, and then giving that back to the human a day or two days later.

12:49And so you can talk about how the Swarm concept was really invented, pioneered, and what it means for the enterprise. Yeah, I think I also want to underscore a very practical example of why this makes sense, right? Like I've been leading and working with engineering teams for a decade plus. And what we've constantly seen is that it's not really talent, even for humans, right? It's not really talent or the technological limitations that delays projects more than someone would budget them or that even causes humans to be relatively slower. It's really the cost of communication. It's the fact that humans don't typically work weekends, for example.

13:26It's the fact that you have to go to multiple teams and multiple teams have different incentives, even within the same company, even if the company is aligned on the same mission. But if you look at the swarms of AI agents that we know will be technologically better. We've seen that happen for the past few years. And if you roll with that assumption, then that system has already solved for the cost of communication. They don't take weekends off. They're constantly working. At the moment, maybe not smarter than an average human, but they will soon be, right? So because you've eliminated that, you can just get stuff done way faster.

14:01And once you've crossed the chasm, as you mentioned, once you've gone from the assistant era to the agentic era, you've adopted that, you've positioned agents in your enterprise, you are positioning yourself for what's coming next. Because as of now, we're thinking of agents and or like swarms of agents as operating, really, we're thinking about it in human terms, where I can hand off a task to someone and they come back with it. Like that's like a human modus operandi. But with the agent swarms, and once you've perfected this process,

14:36to agents. Like customer support, for example, I think is a very good use case just because you have voice agents, you have chat agents, you have the ability to respond to emails, you have the ability to process documentation and understand it. And many times, like the Anthropic CEO Dario Modi said, like humans also hallucinate and sometimes humans can hallucinate more than agents. So the problems are similar, right? And it's really hard for a human to keep context of like changing documentation, changing processes across the life cycle of the enterprise. So AI is like very strongly positioned to like automate away entire systems for enterprises and adopting the agents is a starting point to get there.

15:19What is this? How does this shape the design of engineering organizations in the future? What's the combination of, you know, superpowered 10x coders or 100x coders who are using these tools as assistants still or as sort of, you know, collaborators, plus, you know, teams of agents or swarms of agents that are working, you know, largely or entirely autonomously, plus some sort of new type of engineering management role that's coordinating between these. Like, what does that all look like? And maybe, you know, on a one-year timescale and then on a three to five-year timescale. I'll tell you what the best organizations are doing right now, because there are organizations that are on the edge of AI adoption that are going to be what most of the industry looks like in 12 months.

16:05So we're seeing this right now where you'll have a senior architect, right? That has a system level view of exactly what is going on across the organization from a code-based context and from a business context. They can finally have the code-based context because Blitzy will pull everything together for them and put it inside of a document, right? And they have the business context because they've been there for a long time. That senior person is so valuable, but that's the person that you're turning from a 10X to a 100X, right? That is the person that is expressing intent to this system of action that is going to give it to the swarm of agents, do a lot of work, right?

16:39And you're magnifying everything that architect otherwise would have done if they had infinite time, right? They are expressing exactly what they want down to the nuance level with the help of the system, sending it to the swarm, and then getting back the work, which in this case is code, right? And so we have the senior architect, and then you usually have a couple of junior people. And this is where everyone says, like, oh, all the junior engineers are going to go away. There's going to be no work for them. Those junior engineers that are, you know, two, three, four years of experience now have a place where they have access to understanding the code base, right?

17:10And they have access to tools to level them up. And they can finish out any remaining work that the AI system was not ready for. And people think that, okay, so, like, the engineering org size is going to go down. But engineers have been automating away the work that they've been doing for decades, right? Like deploying something used to be a whole team's job, right? Like now you can get it done with a little bit of a configuration and a click of a button. So engineers are exceptional at automating away the work that they used to do so they can work on the next level of abstraction that the business needs.

17:45Because these engineers are just, they're problem solvers and software is the tool that they're choosing to use. Now AI magnifies their ability to create that. Yeah. And given how broad this spectrum is, right, there's always a use case where you can deploy your engineers to one task or the other and the tools, right, use them optimally. Like organizations and enterprises do not realize this often enough that they always have some sort of a key band risk, right? And this is like the 10x architect or the person who's been around for like 10, 15, 20 years with the company that knows things that no one else does.

18:15And when every time you bring an engineer on, you hire a new team, a new engineer, a team of engineers, you keep that person in the meeting and you help them ramp up. Right. But what happens if they quit and you don't have that knowledge captured? And the AI tools out there, the swarms of agents can help you document and capture exactly that. And that is a decision that you need to take consciously that you make sure that happens. And then there's a bunch of use cases that we come to. Right. Like if you have these modernizations, if you have tech dev, which every enterprise has, if you have documentation challenge.

18:46It makes sense to have the swarm of agents take a stab at it, get it done to the best extent possible, and have your junior developers even finish the job, right? Like Blitzy is, for example, an 80 % to 20 % in terms of time, right? So Blitzy makes you five times faster, but the code that you get is not perfect. So you can have junior developers spend time on finishing that if it's a large-scale modernization. If you have something else where you have a product manager who's not very technical and they want to build a prototype, You can use the appropriate tools for that. So there are enough silos and specific use case across the entire enterprise that lets you use a host of tools.

19:24And there is work to do for every single skill level within the enterprise. Do you see that evolving, though, as these systems get more advanced? Is there a narrow band of time in which juniors fit that role? or do you anticipate because there are going to be kind of continued, as you put it, you know, higher orders of abstraction, there will always be something that sort of, you know, that you're going to want people to augment the AI with? AI is never going to invent something. Well, until AGI is here and then it can learn things, right? But like humans are like brilliant, right? And when you let them focus their time on net new problems that haven't been solved, whether they're 22 or 52, like they will directionally move the organization in the right direction.

20:13So the type of work that that 22 year old is going to do is going to be different, but they're going to be empowered to do so much more because they have system level understanding and a fresh perspective. So like as long as that organization has problems that can be solved through automation, through software of which the backlog of any organization is decades long, If they were to actually sit through and work through what are the things that we would have from an optimal software perspective, they usually just catalog it at three years and then call it from there. And so there will be a next level of work that these organizations are going to need these really bright, talented, young people to do.

20:53We don't even know what some of that work is yet. But like there might be an approach where an organization says, great for a one or two year time horizon, I'm actually going to cut head costs, like do the kind of private equity role and get as much free cash flow as possible. But that will not be an enduring enterprise. And the organizations that are going to like make it in the long term are the ones that are going to understand like people are incredibly valuable. We have to adopt AI across all of our systems so that our people are working on the hardest possible problems that AI can't solve.

21:23So for the best organizations, the answer is they're going nowhere. And, you know, I can speak for this personally, just because I joined NVIDIA straight out of college and I was there. NVIDIA is the only other employer I ever worked for. I was the youngest senior architect from my batch, right? I started as an intern and I really saw the journey to get there. I think we're talking about how junior developers and we're like saying, you know, they probably are not going to be relevant. Like that's what some people are saying. But I think that's completely flawed. What we're going to see in a few years from now, and it could be like two to three years, is that this paradigm completely shifts.

22:00I would, in fact, say that if I had the tools that these people have access to today, like, you know, like the ability to process large amounts of logs and understand where the error could be, like, that's where I spent most of my time as a student developer. The ability to review a document of an entire code base that was written by someone else, and that is an AI, right? And I can follow up and ask it questions and get to know things that I otherwise wouldn't have caught if I did like 10 tickets. That's like game changing. We're not seeing enough of that just because college graduates go through this four-year process.

22:37And that four-year process in the last four years in particular has completely, like the technology landscape has completely transformed. So hardly anyone has knowledge of AI or how to use these tools. but the universities are adapting and people are realizing this. Junior engineers realize this. The first thing they upskate on is AI just because the market is gravitating towards AI engineers in general. So what we're going to see is junior engineers who are able to become a 10x engineer far too quickly. It took me three years to become a senior engineer, but I think we're going to see junior engineers that become seniors in like one year or less.

23:12Today's episode is brought to you by Superintelligence, specifically agent readiness audits. Everyone is trying to figure out what agent use cases are going to be most impactful for their business, and the agent readiness audit is the fastest and best way to do that. We use voice agents to interview your leadership and team and process all of that information to provide an agent readiness score, a set of insights around that score, and a set of highly actionable recommendations on both organizational gaps and high-value agent use cases that you should pursue. Once you've figured out the right use cases, you can use our marketplace to find the right vendors and partners.

23:46And what it all adds up to is a faster, better agent strategy. Check it out at bsuper.ai or email agents at bsuper.ai to learn more. Today's episode is brought to you by Plum. If you build agentic workflows for clients or colleagues, you need to check out Plum. Plum is the only AI-native workflow builder on the market designed specifically for automation consultants, with all the features you need to create, deploy, manage, and monetize complex automations. Features like one-click updates that reach all your subscribers, user-level variables for personalization, and the ability to protect your prompts and workflow IP.

24:21Make your life easier, your clients happier, and your business thrive with Plum. Sign up today at useplum.com. That's Plum with a B forward slash NLW. Today's episode is brought to you by Vanta. In today's business landscape, businesses can't just claim security, they have to prove it. Achieving compliance with a framework like SOC 2, ISO 27001, HIPAA, GDPR, and more is how businesses can demonstrate strong security practices. The problem is that navigating security and compliance is time-consuming and complicated. It can take months of work and use up valuable time and resources. Vanta makes it easy and faster by automating compliance across 35-plus frameworks.