

22 Oct 2026 · 51m · 21 chapters

Blitzy’s approach to autonomous software development for large enterprise codebases, focusing on why current AI coding agents fail at scale and how Blitzy uses a pre-built knowledge graph, multi-agent orchestration, and reviewable plans to produce correct pull requests.
Sid Pardeshi, CTO and co-founder of Blitzy; previously at NVIDIA (since 2016). Kevin Ball (KBall), VP Engineering at Mento; independent engineering coach; co-founded two companies and served as CTO; founded San Diego JavaScript Meetup; organizes “AI in Action” via Latent Space.
LLM context windows can’t cover millions of lines reliably; Blitzy pre-runs for 24–48 hours to build a knowledge graph as a shared source of truth. Agents query the graph (including semantic summaries/embeddings) and follow an approved spec with built-in review, security, and testing checkpoints. Multi-model routing reduces “scope creep” and convergence errors; PRs can be refined with “Refine PR” while preserving graph/SCM history.
enterprise-scale codebases (tens of millions of lines), deprecated/unmarked code found via graph context, and PR review that highlights key changes (not hundreds of test-only files).
Written by AI. May contain mistakes.
Sid discusses his background and the inception of Blitzy.
“Kevin Ball, or KBall, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders.”
Exploration of how Blitzy differs from other software tools.
“And what's the, don't give me all the secret sauce, but like, what's the unique take?”
Discussion on the unique challenges faced by enterprises in coding.
“I want to dig into a few pieces of that.”
Insight into how Blitzy ingests and processes codebases for efficiency.
“And that logic or that pattern is buried deep into millions of lines of code, right?”
Learn about the impact of open source graph databases on coding agents.
“But I think, you know, I've been experimenting with an open source graph database tool for my coding agents, and it does make a huge impact.”
Understand how agents interact with codebases and the importance of context.
“and is actually able to pull that context reliably and make it helpful for the agent and not actually distract it.”
Discover how memory is earned and the importance of learning from commit history.
“So to really deliver value, you need to be able to understand and interpolate instructions that were never provided.”
Examine how agents are coordinated in swarm systems without a central orchestrator.
“We run all of this processing in parallel and use multiple coordinators at different levels to achieve the shared goal consistently and reliably, right?”
Learn how Blitzy automates the code review and security processes dynamically.
“I'm assuming that's some sort of either metadata or a distinct type of node attached to the code node in some form.”
Explore how Blitzy streamlines the review process for faster approvals.
“Maybe there's a design gap that no one else saw.”
Learn how to improve efficiency in code reviews and approvals.
“You're focused on getting this to production.”
Explore strategies for keeping mental models up to date with AI-generated code.
“So that kind of raises another interesting question.”
Discover effective documentation and testing strategies for codebases.
“It's a living document of what that code contains.”
Understand how to deal with legacy code and its complexities.
“I think my favorite definition of legacy code is untested code, right?”
Learn about the correction structures in AI-driven software development.
“It's like 700K tokens and it's done the work.”
Explore how to define project scope and make decisions with AI models.
“And this is a problem that anthropic models have been more vulnerable to, but I've seen it on GPT models as well and other things, which is, you know, agents love to find one more thing to do.”
Understand the process for selecting and evaluating AI models dynamically.
“And all of those models, you know, are specialized at the task that they do, and they've been proven to be the best at that particular task, right?”
Learn about the new private sandbox offering from Blitzy designed for enterprises.
“Yeah, let's talk about sandbox, right, which is the new plan we have.”
Explore the evolving role of software engineering and autonomous tools in enterprises.
“So let's maybe now shift and look forward a little bit, right?”
Discuss how Blitzy manages operational environments and enhances security.
“Actually, one thing you mentioned there that I want to dig a little bit.”
Understand the challenges and solutions related to vulnerabilities in code generated by LLMs.
“And so therefore, we believe like that is the place where the industry is going.”

Get the gist, explore an idea or ask about something said.
On Software Engineering Daily, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Codebases and enterprise organizations often span hundreds of thousands or even millions of lines of code. In many cases, changes must follow established patterns precisely, or risk introducing costly security or operational issues. A common challenge with AI coding tools is that they struggle at this scale. LLM context windows cover only a fraction of a large codebase, and output quality degrades the longer an agent runs. Blitzy is an autonomous software development platform for the enterprise. It starts by understanding your entire codebase, building a knowledge graph that acts as a shared source of truth.
0:41From a well-defined spec that engineers review and approve, it orchestrates specialized agents that work in parallel to build, review, and test code autonomously at scale. The result is a pull request that engineers can then review and merge. Sid Pardeshi is the CTO and co-founder of Blitzy, and he previously worked at NVIDIA. In this episode, he joins Kevin Ball to discuss why large code bases break today's coding agents, how a knowledge graph extends what models can reason about, how a reviewable plan keeps specialized agents aligned, and where autonomous software development is headed. Kevin Ball, or KBall, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders.
1:28He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc.
1:56Sid, welcome to the show. Hey, Kibar. Thank you for having me. Just to quickly introduce myself, I'm Sid Pradesh, CTO, co-founder of Plitzy. I've been at NVIDIA since 2016, you know, back when NVIDIA's adjusted market price was a few cents. And it was not the trillion dollar company that it is. It was nowhere near that. And when I told people I work at NVIDIA, they said, NV Media. You didn't get a job at Google, huh? Yeah. So NVIDIA operated so much like a startup and it was a fun journey. I lived through, I saw the AI wave happen and I lived through it and I saw how Jensen was a visionary. During that wave, I realized what an incredible opportunity this is for applying AI to the world of software development, particularly autonomous software development.
2:47And that's how Blitzy was born and that's my peer. Well, let's give us a little bit of an overview of Blitzy. What do you all do? And what's the, don't give me all the secret sauce, but like, what's the unique take? Yeah, so, you know, Blitzy is an autonomous software development platform. So people are used to platforms like CloudCode, Codex, which are, you know, more tools or even tools like Devon or Factory. The way we're different is that we specialize on very large enterprise jobs. Unlike the other platforms, we are enterprise only, and we have enterprises designing their entire SDLC around us.
3:22So what happens with the other tools is that they're really great at taking bite-sized chunks of work that humans can delegate to, right? You give them tasks. Many of them position themselves as junior engineers or interns, like Devin, for example, right? Cloud Code, for example, is a terminal tool that you can run micro projects with. But the vast majority of the work at the enterprise, about 80 % of it, is well-defined work, especially with legacy enterprises, with very large-scale enterprises. It's very well-defined work that was reviewed by a designer, that was reviewed by an architect, was documented.
4:01it's ready to pick up you know exactly what the acceptance criteria is you know exactly what the process is to get from development all the way into production right and it's just sitting there waiting to be done the humans that need to work on it they're juggling across models you know it's so unpredictable like opus 5.5 just released yesterday and it solved some of the problems that opus 5 had but opus 5 a lot of people struggled to work with it it changed so many things about the model gpd on the other hand has a different product right so what you have is engineers find themselves optimizing their prompts building their skills hopping between tools trying out the different models that come out finding new ways to work with them finding fit between the features and the prompts and the tools and workflows etc so the other tools they've changed what engineers are doing but they haven't changed the fact that engineers still have to do things and sit there and babysit the tools and the agents to get work done where we're fundamentally different is you take that spec you have you give it to blitzy you press build and you're done right you go and do other things you go and use the other tools you go and do other more important things and blitzy does all of the work for you correctly exactly as a person who's been on your team for like 30 years has been doing right it understands exactly the right way you implement things because it does this batch learning across your entire code base when you're on board understanding and mapping out all of the patterns relationships you know identities across the entire code base and then applying those at runtime using the right model for the right time so we use office we use gpt you know we use a large variety of models and model sizes and we always deploy the right model for the right job it's all about for us we're not optimizing for cost we're optimizing for quality right what we want to do is when you get the pr back it is correct it is as close to correct as possible.
5:56And what our customers have seen is that majority of the times we are at least, we've saved 80 % of the time that you needed on the project to get to production. And the remaining 20 % is often because we didn't have some access, you're not comfortable giving us that level of access, we didn't support some integration, or there's just other processes or policies that your team follows before you get something to production. So we map against production, we're enterprise only, and we focus on doing all of the work one shot. That's how we're different. Awesome. I want to dig into a few pieces of that.
6:28But first, let me say I love the focus on quality. This is the thing I constantly challenge my team on in my day job of like, hey, a lot of folks are using these tools to pump out a lot of things faster, but at a lower level of quality. How do we do the same amount of work at a higher level of quality? What do we do to up the level with it? Let's maybe start a little bit with what the enterprise only ends up meaning, what you get to do differently because of that. So if you look at an enterprise customer, what differentiates them from selling to a developer or a small company? Yeah, very good question.
7:01So I would say there's two fundamental things, and we'll talk through more as we identify them. one is that the challenges and the kinds of code bases and enterprise is looking at are very fundamentally different than someone who's working on a hobby project or vibe coding right so one is the size two is the expectations and the standards right like if i don't follow a style guide on a hobby project that's fine as long as it works i'm not going to complain on an enterprise if i make a mistake in the style pattern or if i make a reckless change that ignores a pattern that's well established and required or i make a compliance error or i introduce a security vulnerability that you know doesn't take into account all of the constraints you have or even a load you know a load balancing defect right or a performance concern that item if i if i introduced that with my you know shortcutting and not following standards that means material cost in dollars as the code progresses through steps right someone has to spend more time reviewing it someone has to spend more time testing it and then ultimately if it reach if it gets to production and the customer is impacted the business has lost dollars right so the stakes are really really high the complexity is high right it's you're not looking at a 10 000 line code base or 20 000 line code base you're looking at at least hundreds of thousands of lines for a small enterprise and tens of millions of lines for the largest enterprises just for a single product right they have multiple code bases at that scale that they are actively maintaining with you know hundreds of committers and thousands of committers right so the scale the complexity the standards they're all really high the sizes of the projects and the features you're building are also very high but it's a very high tax like for example if i have to make a 20 line change in a code base right i may think that i could get away with let's say a simple agent that maybe it sets thinking to medium and gets the job done right but the requirements of that enterprise is that we've already added the libraries or the logic that does this.
9:21And that logic or that pattern is buried deep into millions of lines of code, right? And we don't want you reinventing the wheel for that. So finding that just at the right moment, and then doing it exactly the right way, that's the hard part, right? That's the big challenge that we're pushing ourselves to solve day and night, and we've solved to the most extent. That problem is hard for humans, right, to map out millions of lines of code and understand where it is. So let's talk then about the start of the process, right? Because you mentioned a little bit, hey, we go and we ingest a lot of what's going on so that you know, oh, what is that one library off in the weeds that should be used for this?
10:04So what does that look like at Blitzy? Yeah, so, you know, if you look at how far models have come, right, models are really intelligent these days. But whether they fall down, they fall down at the context level, they have 1 million tokens of context, which maps to about 100 ,000 lines of code. But here's the kicker, right? Every token in the context window needs to map to every other token by a dot product in the matrix. and there's no amount of compute or you know trick that you can apply that does this effectively on across the entire context window so most of the model providers that you have there philanthropic open ai there they have the effective context up to less less than 300 000 tokens right and they're applying techniques like sparse attention uh you know sliding window attention or something else across the rest of the window so this is also why for example now this is not the official reason but if open ai has a pricing differential at 272 000 tokens when you send more than that it's a long context request you get charged more right but because of that fundamental gap as soon as you send more than 30 000 tokens more than 300 000 tokens in the request which maps to 30 000 lines intelligence degrades so that's why as you run these single agent tools like for example the cli tools or the single agent persona tools quality degrades as they are more long-running right the the encouragement from all the providers is to do short-running jobs now the way we solve this is that we invented a graph database system that maps your entire code base by identifying its relationships and understanding what every file in the code base is doing and understanding the structure like okay this code base has 20 000 files but here are the modules that are structured right here's what this file is doing in this module here are the features that live in different parts of the code base right if i'm looking for authentication these are the functions and these are the patterns that the enterprise is using across the entire code base and we do that pre-run in about 24 to 48 hours when you start working with us so it's not like you turn on the terminal and you start working right just like cloud code we you have there is a cost and there is a pre-run you don't like you know you have to go through that that pre-run but the benefit is we've now mapped out the entire code base and represented it structurally right so this graph database lets us so for example let's say you want to find a pattern in a code base that's 30 million lines right and there's tons of products there's so much noise so Clot, Code, Curse, any other tool they're going to try to use grep or some variant of grep and try and find it but there's so much noise so the agent is going to burn tokens and hopefully find the right one but what we've seen in many of these code bases is sometimes there's deprecated stuff in there that's not even marked deprecated right so how do you know that you found the right item without actually profiling the code without actually running the code without having reverse engineered the entire code base so that's exactly what Pitsy does first but reverse engineering it and saving those markers in the graph database is contextualized right like even memory for example right memory for the other tools are files strewn across the code base you you don't see it because it's in clots projects folder but you can go and see it if you like but then finding the right memory for the right module or file at the right moment that's the hardest part right so by having the graph database represent that structurally semantically and naturally you have a massive advantage.
13:45So the magic of Glitzy is really just making it easier, drastically easier for these models to operate within their effective context window effectively. Yeah, no, I'd love to dig in one level deeper without, you know, asking for any secret sauce on this. But I think, you know, I've been experimenting with an open source graph database tool for my coding agents, and it does make a huge impact. So thinking about that, you sort of alluded to, it's not just, you know, the structure like you could use a deterministic tool compile this to an ast map the dependencies and have that surface that up but there's more that you're talking about here because you were talking about oh we're putting some weights on this we're saying oh this one's actually it's not marked deprecated but it's not actually used so like what are the different signals you're looking for and how do you expose those to the agent so we're looking for we're actually trying to build and run the code right and we're actually mapping out what is the piece of code that's used based on what the agent can observe.
14:43We're looking at not one file, but the entire code base. And at that scale, when you have like 20 ,000 files, identifying which file is relevant and which is not, it's hard, right? So there are, for example, graph database solutions. There are even open source versions of those implementations. But it's really, really hard to solve for the entire code base. And that is what we've invented and solved, right? Inventing and solving and building a system that is reliable at scale and works for any language and is actually able to pull that context reliably and make it helpful for the agent and not actually distract it.
15:22Oh, absolutely. That's what I'm wondering about. And as I said, without going into secret sauce, what does that query approach look like? Are you exposing a full graph language? Is it a sort of semantic tool? What does the agent actually interact with? So the agent has tools that let it, you know, search the graph database. So for example, we also employ semantic summarization. So, you know, we save embeddings of summaries and let agents search that. The agent is writing cipher queries that is traversing the graph. It's identifying, you know, what is the best way to query this. So the agent has an understanding of the graph database's schema, right?
16:08And it knows the kind of information that it can get and the kind of information it can save in case it finds an inconsistency, right? If something changes, what happens? How do you update the graph? That kind of stuff. So giving the agent access to the schema as well as access to the kinds of queries it can make and tools it can call, that's what makes all the difference at inference time, right? Yeah, awesome. All right. So that's anything else interesting from the kind of ingestion process? How are you pre-processing this code to give your agent like full knowledge that we should talk about?
16:42So I think memory, as I said, is one of the most important items. What happens with memory is in today's world, memory has to be earned. So you have to have used the product to be able to earn that memory, right? And then save it. And so you have to get hurt to know how not to get hurt that way. but magic happens when you're not able to get hurt in the first place because you learned from someone else's memories right so a few ways we approach this one when we look at the commit history and we try to understand what journey has this code been through in getting here right and how can we imbibe that and follow that process so that when we write something it lands on the same planet that the customer expects right the others is the customers often use agents.md skills.md cloud etc learn from those apply those but again identify the context of when they are actually relevant right so saving that information understanding the tech stack summarizing that compounding that over multiple levels that starts paying dividends as you start going right you can make fewer mistakes relative to the customer's input because, you know, to be fair, humans and or users are not the best at giving it the most clear instructions.
18:03All right. So to really deliver value, you need to be able to understand and interpolate instructions that were never provided. Yeah, that's super interesting. I've done some amount of like, oh, let's analyze my previous sessions. What do I get out of that? Things like that. But you're looking across essentially their entire history of this project and saying, oh, where were the things that had to be reverted? What were the missteps? What was the journey? So during that process, can we talk a little bit about like, what does the, are there like certain sets of signals you're looking for? What is the compiled version of this look like?
18:37You know, is it coming down to a set of guidelines or is this another graph of searchable practices? Or like, what does this look like? We're embedding this into the actual graph. So we try to replicate the structure of the entire core base. and the problem that you have is, as you correctly described, you can create multiple graphs, right? You can create a graph of the commits, commit history, you can create a graph of the file structure, you can create a graph of the folders, et cetera. But the problem is, how do you tie all that information together? And then how do you provide that to the agents in a digestible format where they can operate efficiently, quickly?
19:13What we're doing is, we use a structure that maps to the structure of the code base and then we actually process all of those commits you know as much as we identify are relevant and we apply the learnings from those commits to the nodes of the graph so you don't have multiple graphs that you're trying to you know deal with you have all the information contextualized and another unique aspect something that we perfected really early on where the rest of the industry is about is it has yet to catch up it's not even being discussed is running agent swarms without a coordinator so the existing model is you can run you know swarms with there's one coordinator and tens of agents right but there's not in fact research current research has proven that if you run multiple agents in parallel it's really hard to ground them and to get them to not overlap and do things, you know, solve the shared goal because the other agents can have multiple opinions.
20:16We run all of this processing in parallel and use multiple coordinators at different levels to achieve the shared goal consistently and reliably, right? So there's a different agent, for example, that's focused on crawling the graph. There's another agent that's focused on integrating that just in the right place and handing off to the next agent, you know, at the right point. So that I think is one of the hardest problems that we had to solve early. Yeah, so let's dig into that a little bit because as you highlight, right, you can have a single centralized orchestrator, but that breaks down fairly quickly.
20:50How do you deal with collisions, ownership, handoffs? Like what does that look like in your agent swarm? So we use the graph database as the source of truth because that changes everything. So what happens is if you can use, for example, disk, but disk is volatile, right? As soon as I change the folder, as soon as I kill the pod or destroy the location, it's gone. You can use Git, but you could destroy the beauty of the commit history. And you can have a very cluttered workspace that makes it incredibly hard to find information. It's one thing to dump information. It's another completely different to retrieve it intelligently, right?
21:28So the graph database helps you do both by structuring it on the graph, by contextualizing it at runtime and then structuring it on the graph. what you can now do is use that structure to recruit agents so for example if your graph is based on folders you can assign agents to folders and limit scope once you have a scope document you can break down the scope and then distribute it based on a structure that's anchored on the graph so what that does is it couples us to the graph right whatever we do has to be on the graph which is a trade-off, but the benefit is that you can run as many agents as you like.
22:08There is no theoretical limit to how many agents you can run in parallel as long as you solve the compute behind. So let me dig into that a little bit more. So we already talked about when you do your analysis of history and things like that, you're attaching that to the graph. I'm assuming that's some sort of either metadata or a distinct type of node attached to the code node in some form. so now you're saying okay we're going to also attack tasks or desired transformations or something like that also to the graph and now those are claimable and and agents can just come in and say oh i i can do this part i can navigate this does that require you to deal like what is the synchronization primitive oh i have a dependency here i need this part of the graph to be complete before that can go like does that mean i have to claim this whole piece or does that mean I do this and then someone else can claim that?
23:00What does that end up looking like? That is another hard problem we had to solve. So we have dedicated steps for synchronization where if you've completed... So all of this is pre-planned. When you're running a job, first figure out what is the review process. So for example, with any other tool, you have to... Claude just finished working on the tool. Now let me do a code review. And then, okay, the code review is done. Let me use... Let me do a security review. A security is done. Let me run a test agent that tests everything. I have to ask it to do this every time, right? It's not going to do it on its own.
23:31For us, we do that out of the box by default. You don't have to ask Blitzy to test what it did. And again, whether or not it needs testing is completely dynamic. That is also agent determined, right? So by deciding what is the path to production, right? By being an enterprise-only product, what you can now do is just focus on production, right? And then you decide what the checkpoints are, what the stages are for synchronization, when you should do it, what you should be checking at that point and what are the expectations and acceptance criteria at each of those stages and then you can just drive the agents through it until you know you have something that is shippable at which point you can hand off to the human yeah so let's let's walk through that process a little bit more right so if i think about you know and and because i'm not deeply familiar with blitzy i'm like mapping this to my mental model of an orchestrator so So, right, I take, let's use the example you use.
24:24I have an orchestrator handling my, you know, handing things off to Cloud Code. It says, hey, go and implement this. Okay, I got back. It's implemented. Hand it to a reviewer. What does that review phase look like for you? Is it also somehow tied into the graph? Like, what is, do you have to have versions of your graph? Because now you're saying, oh, this is in this branch. It's still under review. Like, what does that end up looking like? Yeah, so we anchor on the SCM for progress. progress and so that is the progress in terms of the work done and there's the agent work stream that is in flight right and there's a scope of the review of the you know based on the expectation of what should be complete at this point so the review as a process is anchored on the graph the results of which are committed onto the scm and so throughout the process you know when you when you have a workflow that needs to be run by the agents, that is mapped and tracked onto the graph.
25:23And again, the results are captured into the SCM. So in case you pause the system at any point, you can reconcile both and know exactly where you are in the overall journey. And does that then continue on through, you mentioned security review. What does security review look like in this context? So security review, the thing is, you don't want to be writing any code that introduces security vulnerabilities. with the mythos scare that everyone had, especially because we're working with banks. You have to guarantee that the code that Plitzy writes is free from issues. So by default, you have another set of agents watch the works that are being done, catch security issues, and schedule security checkpoints to make sure that, let's say, those issues were temporary.
26:07Maybe it was a scratch that the agent wrote and it plans to discard that later. So let's make sure that happens. and that gap doesn't make it into production or two when we plan this feature you're introducing a new api i got to make sure that it follows the existing authentication patterns that it you know it follows load balancing and follows all of the core items that you follow you know not just from security but also from a housekeeping standpoint right um so let's do that testing let's do adversarial testing let's try to break the system and product before someone else tries to do it and let's flag that.
26:40Maybe there's a design gap that no one else saw. So it's Blitzy's job to make sure that it exposes that. If that item is out of scope, it documents that. If it considers that this should be addressed as part of the same job in scope, it would go ahead and adjust that. Documenting is interesting. I think one thing to map back to a human process, as you're doing work, you often uncover future work. You're like, oh, this is out of scope for this project i don't want to explode my scope i don't want to explode my surface area but like this i found this we should really fix this yeah does that end up getting resurfaced to a human to decide does blitzy schedule itself future work to do those cleanups like what does that end up looking like yeah you're onto something here so we're working on the ability for blitzy to file tickets and schedule itself and to be given a goal and you know recursively self-improve itself to achieve that goal.
27:37But what we do at the moment is at the end of Glitzy's work stream, we have two things. One, a project guide that documents all the work that Glitzy did and all the work that it could not or decided not to, right? So that includes these findings. And that also affects the completion percentage if it was an in scope item. The second item is the PR review experience where it walks you through all the changes that it did, but just focuses on the key changes. So you don't have to review 500 files. You can review like 50 files and you understand the work that was done because if I have 100 files just with unit tests and you know they're all green and you know how they were written, you don't need to review my unit tests.
28:17You're focused on getting this to production. So using those two items, a user is able to feel more confident while approving the changes, reviewing it and move faster to the review process. So that kind of raises another interesting question. I feel like one of the things that many of us are struggling with right now with AI-generated code is how do we keep our mental models of our systems up to date, right? It's great to have it working, but if I want to understand how I can evolve the system well, I need to understand the underlying structures. And I think in an enterprise setting, you have a lot of existing structures.
28:57As to highlight, Blitzy is very good to respecting those. But sometimes things need to evolve. Sometimes things need to change. And especially if you have a long lived code base, there may be pieces in there where like, hey, this is hairy. We don't want to build in, even though everything is depending on that new code should not look like that. We want to shift our model. We want to do things. So kind of what, how do you handle the like, keeping the brains of the humans up to date with what Blitzy is doing for them? Yeah, there's multiple, you know, strategies. So for example, especially we run into that, use case a lot where you know i have this but don't don't do that that was that was a dark past i don't follow that so again even when we look at comments right it's about recency you may have one pattern earlier on maybe years ago and you maybe have moved on to a new pattern or a new technology framework more recently so recently recency is weighed into the you know factorings of that agent that applies patterns but other than that we have knowledge base which is one of the features, but as you onboard and give more code, it documents all of those branches and its understanding of those branches via xpec.
30:06It's a living document of what that code contains. You can look at it and as you update and resync code, that document gets updated. If you need to understand what any branch in your entire code base does, you can look at that knowledge base and you can understand it. Then the more tactical items that our forward deploy engineers work with customers on is creating documentation and keeping that documentation up to date using blitzy right so what is the current state what are the best practices before running a massive project what makes the most sense is we follow this troll walk run approach where we suggest that you start with documentation let blitzy confirm its understanding of your entire code base let us reverse engineer and let us show you what that looks like move on to adding testing right because how do you validate the work that Plitzy did or anyone else did, right?
30:56So let's increase test coverage and let's add that. And then what those two things do is now it's very, it's much easier to describe acceptance criteria, right? It's much easier to explain to a human what that code does. And it's much easier even for Plitzy, once the human has actually reviewed the documentation, to validate its understanding and make sure it's on the right track. so by doing that and then again if there are any caveats by capturing that in the agent action plan sometimes there are conflicts to what the human thinks is in the code and what they ask us to do or even in their own prompt right so flagging that early and making sure that's addressed like some of the tactical things that you can do to make sure that the cost of this item is reduced yeah that makes sense well it's it's interesting again with the enterprise context because you know long lived companies often have parts of their code base that are poorly understood at best, maybe behave in odd and unexpected ways, but then there's downstream dependencies.
32:00I think my favorite definition of legacy code is untested code, right? And so this idea of, hey, first, let's validate our understanding, both with documentation, but like, you may not have anyone who can verify that documentation. So let's write tests that show what the system actually does. Yeah, I think our favorite moments and the customers that love us most are the ones who have a lot of legacy code and a few people who can understand it. So it feels amazing when you have architects who have been there for like 35 years, you have COBOL developers who no longer write that and having them validate the code and really express satisfaction that it's able to understand and operate in a way that they would.
32:44I think that's the best part of building and working with Blitzy. Let's talk about another thing you mentioned here, which is how course corrections happen, right? No matter how good the system is, there will be mistakes, there will be stuff, and you have this human review process to check and verify. Let's say something goes wrong. What does the correction structure look like? How does that get parsed, represented, et cetera? Yeah, so I think you bring up a very important point. And part of my frustration with, you know, working with other tools has been that you blew up all of the context writing the project.
33:17It's like 700K tokens and it's done the work. And now to do the next task, let's say you have to compact. So what happens is that it forgets many times the work that it is, even though you have the plan, right? The plan is like a two pager. The work is maybe tens of thousands of lines, if not hundreds of thousands of lines. How do you reconcile that, right? So again, because all of the work that Blitzy does follows the graph, the graph that it updated on the fly corresponding to the branch, we have a massive advantage. The way we operate is you saw the PR, we found some gaps. We have a workflow called Refine PR.
33:56And what that does is that it resumes the process, including the graph. It preserves the history, preserves the history from both the graph and the SCM, and addresses your feedback. And the best part is that we bill on lines of code, right? So we don't charge you for mistakes that we make. Everything that you give us feedback on is a mistake. So what that means is if by addressing your items, lines of code in your resulting PR went down, you will only be charged for the final lines of code. So let's say Blitzy went off and wrote 500 ,000 lines of code and when you were only expecting 300 ,000.
34:36Sounds like a plot project. recent cloud project although i hope they fixed it with the recent models you found that 200 000 lines were just comments and fluff that you don't agree with that you didn't ask for fine you know you can ask it to clean that up and then you only get billed for the correct answer so that i think is very unique and special for blitzy and i think is the right model for the enterprise because there's a lot of token anxiety these days the tokenomics and the fluff of you know let it rip burn all you can is all gone right enterprises know that they need to be cost conscious with these things because if you let it run wild it's going to the wheels are going to skyrocket and i i believe we position our product and our pricing exactly with that idea in mind yeah i like that a lot because it aligns all of your incentives so it's like let's write the right code not all the possible code a couple of things that that actually brings to mind so So one is sort of the convergence problem.
35:38And this is a problem that anthropic models have been more vulnerable to, but I've seen it on GPT models as well and other things, which is, you know, agents love to find one more thing to do. So you go, you build a thing, review it. Oh, you haven't thought about this thing. Okay, right. Okay, I'm going to go and build it. Okay, you haven't thought about this. And eventually it's like telling you, hey, once every Tuesday in July, like this is going to break in this way. So how do you, within Blitzy, say, this is the line, this is what's enough, we're done? I think you highlighted a very, very important problem.
36:14It's not just anthropics, it's a problem with a wide variety of models. And people believe it can be solved, prior art thinks that you can tweak it with, let's say, prompt tuning or with effort tuning. Hey, if you think it's doing too much, just drop the effort level. but it doesn't really solve the problem right because the way these models work if there's one thought in there that is going to let it to think about finding new problems or have it document like one thing it found you can't go back and change that unless you really edit the conversation which is not something a regular user does right so it's a really hard problem but the way we solve it is at any point of time we use different models the beauty of using different models is that you get fresh perspectives and you can align on the stronger model to be the tiebreaker right so we have internal evals and benchmarks we know which model is relatively stronger for decision making or which model is relatively stronger for front end and then so on and so forth right so by applying that at runtime so even if Claude thinks that two things worth flagging or this item is load bearing in typical Claude speak you have a gpd come in and check and say, nope, that's not in scope.
37:29That's not what the agent action plan says. And GPT is more effective because it's doing less, right? Cloud has the burden here of implementing everything. It has blown up its context. It's operating probably in the ineffective side of its context window, right? It's loaded up with more than 300k tokens. GPT in this case was only looking at the AAP. And because it had to deal with much less constraints, it did a more effective job of identifying the scope. So this is a very vague example, not what we actually do, but we do something very similar across many models. And we arrive at the correct answer to what the scope should be.
38:07So before the user sees it, that job has been seen by at least eight different models. And all of those models, you know, are specialized at the task that they do, and they've been proven to be the best at that particular task, right? So imagine switching across eight different products locally to do one task, right? That's painful. It's not something you want to do. And you're going to have to maintain your environment and prompts and instructions for all of them, right? You don't have to do that because you use Pitsy. So that is interesting and a nice segue into another question I had. So your multi-model, which is great.
38:42I think that's definitely an advantage being independent from the... Right now, a lot of the harnesses are like tied to a particular model set and things like that. So being independent, letting you pick and choose is phenomenal. How do you decide which models to use for what? I think you mentioned internal evals. Like what does that process look like and how does it evolve? Yeah, so we work very closely with labs, model providers, you know, inference providers. And whenever a model is launched, we run it internally, benchmark it. We run it in the platform. We test how it's doing. We score it, right?
39:15And then we apply those learnings to decide which is the tiebreaker. But at runtime, the process is entirely dynamic, right? We have these ladders and rungs. We have dynamic selection of the models. We have switches and routing in place based on capacity, you know, based on intelligence, based on task specificity. And the agent chooses the LLM for the next agent based on the task. It chooses the system prompt. It chooses the tools, right? So this is literally agent designed by agents, right? And what happens is as the agents and the models get better, it pays dividends because they're more precise.
39:51They're more effective at selecting the right model for the right job, right? So to build that engine, it took us years, right? And now we've perfected it. So it's really, really good at defining even the guardrails, right? Here's what's out of scope. And here's where you can check with me if you have questions, right? So rather than the agents are trained to go back to the human to ask questions. but adding these you know adding this framework in there just helps us do autonomous work right we don't have to go back to the human yeah so that's interesting i'm curious about that you know having the agent choose the model which i think is is a really nice you know self-improving approach but what do you use to describe the models to give the agent sort of dimensions on which to choose right like if i just say hey your options are opus 55 or gpt 56 like it doesn't know what those are.
40:43Maybe it does, but what are the features that you're calling out about those models for the deciding agents? Yeah, so we ignore all the fluff that the labs claim about the model. One lab may think Model X or XYZ is amazing at something, right? Or the benchmarks may say one thing. So that's why you don't rely on web search for something like this. But instead, all of the results of the benchmarking and the evals that we had internally that identifies the strong points, you distill that into tags. and you distill that, you rank the tags. For example, front end, you have a ranked ladder of which models to choose based on complexity.
41:22So front end, very complex, choose this model. Front end, low complexity, choose this model. It's good enough. And then so on. So because you're optimizing for quality and speed, you need to choose the fastest model that gives you the best quality. So for that, we've built our system to be intelligent enough to make the right decision at runtime. That's really nice. So you're kind of mapping out what is that Pareto frontier along a number of dimensions. I think, yeah, front end, back end, language, complexity, those make sense. Okay, awesome. Is there anything that we haven't talked about about Blitzy today that we're missing that would be worth talking through?
42:01Yeah, let's talk about sandbox, right, which is the new plan we have. So we wanted to make it more accessible to enterprises, and which is why private sandbox is what we've introduced. And what that means is if a company has more than 250 employees and you sign up with your work email, you automatically get access to Blitzy with a set quota and a significant amount of lines of code to both reverse engineer and to write. It's a decent quota. I don't have the numbers handy and they keep changing, so i won't cite them but it's decent enough to do a small to medium-sized project right depending on what you scope it and you could use that to build a completely new product which is very easy to get started right you don't have to code your environment in or try to get it to understand your code but there's no waiting just get started right away can easily evaluate how good busy is or bring in a piece of your code base we don't train on any of it we don't you know we have very strict data retention policies and all of that so it's an enterprise-grade platform right we're We're not doing anything sketchy.
43:01Your code is owned by you, etc. We don't store any of it. So you can run that project through Blitzy and experience the value, right? When you get the PR, when you go through the review experience. So it's a quick way to expedite the process where you don't need someone to handhold you. You can experience the value and then speak to our team if you'd like to move forward. I think it's a very generous offer. I don't think that's available with any other tool. So we really hope that a lot of other enterprises get to experience the value. And we're really confident about that, which is why we're doing it.
43:32That's awesome. So let's maybe now shift and look forward a little bit, right? So we've talked a lot about what Blitzy does. I think there's a lot of interesting things to pull out of that, whether you're in an enterprise where you can apply Blitzy now or considering or trying to think about how you're managing your own tools. What's like the frontier? What are the things coming down the pipe? Well, I think the frontier is there's two wild visions of the frontier two broad buckets i think there's some people who think that what's going to happen going away is code review as a process is going away software engineering is dying we're not going to be writing any code we're not going to be reviewing anything we're just going to ship i think i i find that personally outrageous especially when you're talking about the enterprise i don't think that's the speed at which the compliance frameworks evolve or that level of comfort is what the architects who actually use the models on a day-to-day basis have.
44:28I think they have a much stronger and clearer understanding of what the models are and are not. And to be fair, they're on the spectrum of glorified autocomplete to AGI. They're not on either end. They're somewhere in the middle. At the end of the day, there are sequence-to-sequence models that are predicting the next token. right but they're really really good at real world use cases right so where does that leave us if you go back to the beginning where i said most enterprise work is very highly defined and can be captured in a spec and that's how enterprises have been building software for generations right we see the opportunity for the enterprise to hand off that work submit the job do something else do more complex discovery work do more complex research work that you cannot hand off the requirements are not clearly defined and the models are amazing at that right so the tools that you have plot code codex etc they're amazing at helping you do that work right and the well-defined work gets done by autonomous tools like let's see right and these tools the next frontier is like recursive self-improvement right so you can set an autonomous system perfect something like i want to ensure that performance for my e-commerce website my page always loads in less than a second i have a very large team they're all shipping tons of commits every single day but i cannot regress on this one metric and it's very hard for me to enforce this with every single committer so what i do is i set the system behind it and this system monitors and makes updates every single day floating up for human review dynamically when it needs it but it achieves that goal right and you can and we see that growing into not just you know distinct goals within the enterprise but to entire products right if you can capture the product objectives and the metrics right and the standards that you want to maintain and achieve then what you can have is autonomous systems like blitzy run these products that are very well defined that only need human supervision once in a while and that can be easily managed right at the same time shift the human attention to more complex items to products that we need to be far more dynamic where you cannot trust an llm where you need research and you need to do a lot more auditing and then compliance checks etc and focus the humans on that so it's an incredible opportunity for every player in the system because the market is big enough to accommodate everyone the software software engineering is such a huge, huge market with such an elastic demand that it's a really incredible time to be alive and be an engineer in the space.
47:20Yeah, no, absolutely. Actually, one thing you mentioned there that I want to dig a little bit. So in this view of almost self-managing or autonomously managed projects, where do things like runtime environments and logs end up? We didn't end up talking about that for Blitzy, but are you feeding back operational data in? How does that end up working? Yes. So Blitzy is connected to the cloud. Your cloud, essentially AWS, GCP, whatever that is. Blitzy has its own environment. So typically when we work with customers, we have them capture the environment variables, secrets, configurations, instructions into Blitzy's environment.
47:56And then Blitzy replicates that environment across projects. You can have micro environments. You can share environments across projects, etc. etc and for for the vision that i talked about right that is the first step of getting to work with the system right get it access to your development environment have it it operates like an independent engineer so have it recreate its environment make sure it's it's good at that and our environment is completely dynamic there's no static item right the agents set up the environment every single time they operate they do it dynamically and they maintain that right So once you've solved that environment problem, once you've given it access to observability, logs, and once you've set goals, guardrails, boundaries, the agents are incredibly good at optimizing all of those simultaneously to making sure they do the right thing.
48:45And if they're unable to do that, to escalate to a human at the right step, to stop and get the humans in. So we do that internally and we've been successful at getting customers to trust us and to deliver value by doing that. And so therefore, we believe like that is the place where the industry is going. Yeah, no, I really like that. And also thinking about things like security, vulnerability updates, all these different things. LLMs have increased the pace of vulnerabilities being found. But I think autonomous systems like this can also then increase the pace of reacting to and fixing those issues.
49:21Yeah, I think the biggest problem that people don't realize with LLMs is that LLMs have increased the pace of vulnerabilities being found. but they've also drastically increased the pace of vulnerabilities being introduced. And that's because you have to prompt an LLM to write secure code. You don't have to prompt a human, right? A human learns it and remembers that LLMs don't learn things until they're trained, right? And I'm sure the labs will continue to work on and get LLMs to write better code, more secure code, you know, but they'll always be, because LLMs get much better at writing code, it's not a lever that you can control.
49:58there's not like write secure code optimized to 0.99 so this is always right secure you can't do that right it's training is very dynamic you can only control some other variables that are not this so because of that it's on the harness and it's on the system to enforce security right and which is why we take that very very seriously and we have an entire team dedicated to this and we have entire workflows within Blitzy, we factor this into the design of how all lines of code written by Blitzy are built. Because once you solve that, once you help the customers get comfort about security, compliance, the guardrails, code quality, that's when you can move to the next level and actually bring in this transformation that you're thinking about.
50:55Thank you.
Codebases in enterprise organizations often span hundreds of thousands, or even millions, of lines of code. In many cases, changes must follow established patterns precisely, or risk introducing costly security or operational issues. A common challenge with AI coding tools is that they struggle at this scale: LLM context windows cover only a fraction of a large codebase, and output quality degrades the longer an agent runs.
Blitzy is an autonomous software development platform for the enterprise. It starts by understanding your entire codebase, building a knowledge graph that acts as a shared source of truth. From a well-defined spec that engineers review and approve, it orchestrates specialized agents that work in parallel to build, review, and test code autonomously at scale. The result is a pull request that engineers can then review and merge
Sid Pardeshi is the CTO and co-founder of Blitzy, and he previously worked at NVIDIA. In this episode, he joins Kevin Ball to discuss why large codebases break today’s coding agents, how a knowledge graph extends what models can reason about, how a reviewable plan keeps specialized agents aligned, and where autonomous software development is headed.
Full Disclosure: This episode is sponsored by Blitzy
Sponsorship inquiries:
[email protected]
The post Blitzy and Autonomous Software Development for Enterprise appeared first on Software Engineering Daily.