The State of AI: Elon’s $1T Package, Apple’s $600B for Trump & How Startups Win w/ Dave, AWG & Blitzy Founders Brian Elliott & Sid Pardeshi | EP #193

27:22What was the causative agent that said, okay, let's build that? What was that founding story like? Yeah, if you can rewind the clock back to the GPP -3 to 3 .5 era, right? Where these things could code, but it wasn't what we're experiencing today. Right? At that time, Sid and I were doing a pro bono project for our favorite local bakery here in Boston as a part of our time at Harvard and They mentioned they're back to spend You know 300 $400 ,000 on a new mobile app, which got our attention some young enterprise in arch for Newers And so Sid and I went home and we did what is now two and a half years later called viperity And we built them the application overnight, right?

28:03But literally over the weekend over the night. Yeah And which, which now is like no deal, but it would be paid that they pay you, you know, $200 ,000 for that. Yeah, we should have acted like it took longer. I think that was our, that was our first mistake. But was so clear during that time is that sit and eye were actually the bottleneck for development. And so we were, you know, we get an error and you get it back to the system and then you give it to a different model rate. And then through that practice, you're able to get much higher quality code. And so we said, if we could just invent a system where all All the commoditized development work could be removed.

28:41And we could have multiple models going back and forth iterally refining and getting to code that compiles and tests. That is going to be what the future looks like. Right? And we learned that by doing, by being hands on, and then having an idea of what the enterprise needs from such experience and building towards that. Well, were you guys already friends or did the all -nighter making friends? Yeah, yeah. Yeah, we've only got, I mean, Sid's actually the Godfather to my son. on Blake or not. So, yeah, it's important. I guess an investment for friends is pretty good. I mean, and that's another part of the story, day that we've talked about is some of the most successful companies are when best friends get together and just build 24 or seven.

29:22It is a, I say this to the entrepreneurs that I coach, being an entrepreneur and having co -founders, you're going to spend more time with your co -founders in the trenches than you do with your husband or wife or kids. It's an intense period of time. and you better pick somebody or somebody's that you love spending time with. Yeah, I always tell people it's imagine you're on a long international flight when those 14 hour flights and you're sitting right next to somebody. Do you walk off the plane feeling good and having fun or do you walk off the plane not waiting to get away from this person?

29:56Your startup's going to feel just like that every day. It can be work or it can be fun. It just depends on the personalities and the match. So Dave, what do you find most exciting about Blitzie just from as investor? And so they came through link studios and link investment. And tell that to a little bit of the story if you would. Well, they're definitely much more experienced than a lot of the entrepreneurs around the studio. So they had the plan and the idea of fully baked on arrival. We were still first money in and still gave them space and support and it made a bunch of introductions, but they already had it more than figured out.

30:32So that's not all the companies fit that profile. They're also very different. A lot of the companies coming right out of dorm rooms will do image generation. They don't understand the word cobalt or PL1 to save their lives. So when I look across the range of business plans that are right in front of us with AI, more of them fit into the need to understand the domain space, then you can just think of it in a dorm room space. There's maybe two thirds, one thirds, rough numbers. And so what I'm hoping with Brian and Sid is that they inspire a ton more people To go after these you know, these are still multi -trillion dollar markets But they're not you know I AI girlfriend.

31:12They're not you know apartment search other photos you think of while you're in the yeah another photo sharing out there They're you know and they get really deep, you know, you've got all these you know manufacturing You know semiconductor manufacturing Automation you know, that's that's really deep you get insurance actuarial risk adjustment that's very deep. This one is actually nice and that it's very, you know, code generation is very broad. So it's a huge market. But it's also deep in that, you know, like refactoring 10 million line code bases is a, you know, it's a pretty deep knowledge set.

31:44The other thing that's that's really cool about Blitzit and me is that we have all these code generation products. So I use cursor with a Windsor, for Eplet, Lovable. Companies are all worth billions now. But I read a lot of code or I tell it to read a lot of code, it creates a button for me. I say, I don't like that button, make this other widget. And you're doing it in real time. But you can't build something really big. And when you put cursor in full agent mode, it's right in no man's land. It sits there in grinds for like five or seven minutes, which is too long to wait, but too short to build something substantial.

32:18So they're getting stuck in no man's land. Blitzie just said, no, we're going all the way to the other end, where it's going to run all night long or all week long, like Brian was just saying, and come back with something really big. And that's just fundamentally a much different engineering problem than what lovable, you know, replete cursor, windsurf, or doing. It's just a different kind of company. I don't know of any other company that's there. Amazing. Well, we're here today to announce a particular piece of news as well. Some groundbreaking news. Is it Brian or Sid, which one do you want to talk about what you guys are doing?

32:53Yes, please. You are the inventor of the technology here. Sure. So tell us. So every time a new model comes out, they benchmark on this leaderboard, which is called sweetbench verified. And the leaderboard itself was built by OpenAI. It is a subset of sweetbench. It contains 500 problems that were wetted by the researchers at OpenAI. And they confirm that these are solvable problems, and these are worth testing models on. And it's been ubiquitous. So every time in the new one comes out, you always see results. The current top of the leaderboard on the sweep bench website as a filming is 75 .2%.

33:28So we hired a bunch of extremely talented interns. So Dave, going back to Jack, if we could hire him today, we would. That's how good some of these interns are. And every single intern who worked for us, they were amazing. We really credit this to their effort and to Neroj that the efforts on our end, but we ran Blitzy on SweetBench. And as it turns out, these are 12 repositories, but they have 500 branches. And that equates to 400 million lines of code, if you ingest them on Blitzy, right? We've ingested anywhere from one to two billion lines of code overall on Blitzy, depending on how you count it, because you count updates and the whole raw thing as well.

34:11So it's two billion lines of code if you count all of the updates, right? including sweet bench. So we ingested all of that. We ran Blitzion solving the problems and our final result, accounting for everything that we've tested and verified using SB CLI was 86 .8%. That is a significant jump over the current leaderboard on the website. And the last time this was done was when Devon had a 13 % jump from 1 % to roughly 14 .0 some percentage. So we've come a really long way with the system and the primary reason that you were able to achieve this echoing some of the points that were made earlier is we're very different from the existing to the structure, right?

34:54So one thing is you can reproduce these results in production using Plitsy, right? We've not added any custom sacks scaffolding just for Sway Bench, we've not tampered with any of the features as to achieve this. We've seen reports from some of the other labs that claim that even though for example, let's say a latest frontier model claims 80 % on sweet bench if you actually run it and are reproduced it you get 60%. Right. And we wanted to not have that problem. We hear deeply about reproducibility and the practical real world applicability, right, which sweet bench verified has been vetted to be good at.

35:29So you can reproduce these results and it's live as of today. Amazing. Hey, Alex, help us understand how big and important this particular hallmark is for the company and for the world. Give us some background here. Sure. Well, well, first to Brian and Sid, congratulations on your announcement. I think there, I would expect there's going to be an enormous amount of interest from the community once they hear these results in trying to get the results. and in reproducing those results. So congratulations and advance on the onslaught of interests that I expect you will receive. I think to answer Peter's question, I think software engineering is arguably the first major vertical of human labor that is very high economic value, very high productivity that is perhaps succumbing to automation.

36:23So any sort of step function improvement in software engineering is arguably super transformative to the global economy. And maybe just pivoting on that thought, one of the first things that I was wondering when I heard that you would be announcing these results, and maybe jumping back eight months, we all remember when DeepSeek aka High Flyer launched R1 January of this year, and there was sort of an aha moment all around the world. They didn't just announce a new reasoning model. They announced, and maybe this got a lot less attention, they announced a bunch of new open source libraries at the systems level, like a new file system.

37:06So one of the first things that I was wondering when I learned that you'd be making this announcement is, the whole world is sitting on the sort of palimpsest of legacy libraries and operating system code, billions and billions of lines of code, Linux, Python, GNU, all of these libraries. Is there something that you and Blitsey and this new remarkable capability that you're announcing can do to speak to what can we do to improve the performance of this entire tech stack that the whole world runs on at this point? That's a fantastic question. We've been running some of these experiments. We've been taking some of these open source libraries that, for example, it was in MATLAB for one of our customers that they were using and we converted it to Python.

37:56MATLAB was around 20 years ago, it was specific to Windows and we made it worse, Ignostic. We've run these, we've run these Fiosys all the time where we go from OS Ignostic to OS specific to Ignostic from traditional to modern. But we've also been running other kinds of Fiosys where for example, we picked an Nvidia repo. We identified an issue that was marked as open. And we just put Blitzy edit and we saw it. we created a pull request that's all the issue. So if you think about that problem, you can identify bugs, issues, feature requests in any of the modern frameworks and systems and LILIC.

38:30The system is not limited by how much code, how big the repo is. So you can simply add it, it will come back with a solution, you don't like it, you can iterate over it. You can create five projects, get five different pull requests, see how that works and deploy it, all within a matter of days. I think that's a fundamental shift that really is going to change the way people work with open source and also close source technology. Amazing. What is the largest repository of code that you actually tackled?

39:00Brian, you want to say? Yeah, go ahead. So we frequently see 20 million lines, but I think the absolute largest we've seen is about 16 million lines if you want, but it's successful. Crazy, yeah. And that one, just for comparison, for fun, if you had to guess how long would it take in terms a human labor hours did do that. The same thing. It's insane. Brian, I think we frequently scope these by the as part of the PLC process. And I think you probably see to that. So what we do, so there's a question of like how long would it take to grow up 60 million lines of code? And the reality is it's just too big for a human to understand.

39:37So you might pay, you know, Accenture $100 million in three years and they might come back with some some diagrams over the 60 million lines of code. And by the way, the time of the dollars, But the time they did that, what they came back with would be out of date. Exactly. Which is why you can see that essentially the industry has been stuck. This is why your airlines are always misrouted and they can't get their shop or update. This is kind of fundamental problem. So every time we run Blitzy, we actually estimate for the clients in production or enterprise clients how many hours were automated.

40:06And the CIOs love this because it's like the KTI they give the forward on how many hours they've automated away by their intelligent, better selection. But GROCHIN is sort of an impossible problem, but the real value is in the code generation, right? Being able to accurately affect and develop code and accelerate that life cycle for the development team, along that large interest zone, under foreign corpus. But importantly, like Alex, I know you want all developers to sort of go away and we're going to live in a society of abundance, but I think it's kind of going to go in the opposite direction where, you know, it's almost an infinite demand for code, right, and for first -offered development.

40:37And so what's he doesn't do everything? What's he does about 80 % of the quantum of work on average for these large scale problems? But it knows exactly what it doesn't do, which is really the power. And it could have hands off that batch of work to the human developers to finish things out. So we get a really clean full request, plus human labor at the end, to accelerate the development, but not sort of remove the need for the developers altogether. So I think if I may, just to pull in the thread Brian, I mean, I would argue we're about to enter sort of an age not necessarily of just abundance, but of great projects when it's possible to send lots of, basically, lots of automation loose on the world and fix all the problems.

41:21Solve everything, as it were. In the case of Blitsey, this is letting AI agents loose on an enormous sprawling legacy code base and just fix everything. I think it's a good name for the episode. Solve everything. or a book or or or you know fill in the blank. But are you familiar? Maybe you're tracking this project. I love this idea. The great refactor. Yeah. What is that? What's the great refactor? So the great refactor. I love this kind of this is like a classic solve everything concept. This is we've built our whole civilization on a bunch of software libraries that could be better maintained that are filled in many cases with legacy memory vulnerabilities.

42:06There are statistics out there that most of the insecurity of present -day software is due to the way the software is written that exposes them to certain type of cybersecurity vulnerability memory vulnerabilities. If we can only rewrite all of these libraries, that sort of, if you know the meme that goes around of the entire stack being built on, of civilization being built on, just like one block at hangs by a thread. If we could rewrite all of these libraries and dependencies and software supply chain upstreams that our whole civilization depends on in Rust or some other memory secure language, suddenly that would fix almost all that would solve everything in terms of so many vulnerabilities.

42:48So I guess. I think 200 customers sent me that project. Like they just like immediately saw that and it became hot news the day and they all sent it to me. Like, oh, are you guys gonna do this? And I said, well, are you gonna pay for it? Like, yeah. I know. He's the most beautiful thing, right? If you think about this idea where you can give these projects three factors or what not to AI and have it come back with the code, right, you can do that with any chatbot. You can give it to any AI, have it right code. Like getting code from AI is a commodity, right? But if you add constraints to that problem where the code needs to replicate the existing from snarling.

43:25It needs to compile. All the unit tests needs to pass. It should not have newly added security vulnerabilities. And all the other items that are crucial to the enterprise or the problem itself, right, that make it valuable. That's when the challenge is begin, right? And that is not something you can achieve. So, Sid, my question is, I got 3 .2 billion lines of code, which is my genome. Can you compile that for me. Can you like go and identify more, you know, fix the, fix the broken parts? I find that. As long as LLM skin, right, as long as it's in the language that LLM's are trained on understanding, we can do it.

44:06The scale is the problem that we've solved for and the other problem we've solved for is making sure the requirements match. So that when we put you back together or edit you, you actually look like you, right? It's validated that it is you. We didn't change or break something that we shouldn't have. Alex, what do you think about it? You guys should divide them. Yeah, so maybe narrowly on the bio, I, I, there are many other projects that speak the language of the genome and the proteome that I think Peter for rewriting your genome, you'll have the opportunity over the next few years to, to use one of these biological sequence -based foundation models to, to do some variant of that.

44:43I, I do want for, for, Brian and, and Sid though, So really pull on the economics of this. So I really want to press you guys. When we talk about the great refactor, some of these great projects to basically rewrite the source code basis for much of our civilization today. And you think about the economics of that. And there is a school of thought that says we're seeing generative AI hyper -deflate by 10x per year or so, in order of magnitude cost reduction every year. At what point in your minds do you think using Blitzy or maybe competitive tools does it become reasonably economical to basically rewrite all of the legacy code out there that civilization depends on?

45:28I would argue from a value perspective It's there today because the value that this would be provided to society is just dramatic now This is a question of like who's the payer? A line of code from Blitzy is a hundred X more cost -spaces than a line of code from some any provider which is maybe a hundred ex -lessent would be from a human development, right? So we're talking about huge or as a magnitude difference. And so would it be worth it from a society value to rewrite all the software today with what's here? Absolutely. But am I going to continue to serve these financial service institutions and insurance companies first that are readily paying me today?

46:01Like yes. And so I think if you if you grab the capital funding for us to to break even on the say WG, we'll start rewriting all the we're going to write Linux to you just to. I'm going to insert another topic in here. You guys shot a really cool podcast. It's on your LinkedIn, where you're just bantering between the two of you about the fact that the definition of truth within large scale software has always been the functional code. Here it is. This is the final thing. It runs the PL1 code that does all the nav accounting for the mutual funds over at State Street. It's like millions of lines of legacy PL1.

46:33But that debugged code is the core asset. And then the documentation is just something around the edges. Post -Blitzy, the truth moves to the documentation because you can regenerate the code overnight anyway. And so your actual core asset has moved from code to a document, but it's going to move again. And this is where it's really cool to hear you guys bantering around like, what is then the foundational truth of this piece of, because like Alex is saying, the entire infrastructure of society is about to move and also expand 100 or a thousand or a million X because code is so cheap to create all of a sudden, we have much, much more of it.

47:13So you got a much bigger world, but the ground truth is some other format than just P .O. won or Coball or Python code. It's this human readable today spec becomes the central asset. That's a big shift. The specs is still an abstraction layer, right? So that's easy for the human to look at. The real source of truth or understanding is actually, we create a customer -specific hybrid graph vector database that understands exactly what is going on from a functionality perspective. You could change that functionality from one language to another, but we are capturing the core essence of what is required there.

47:52We can display that as a spec, which is 200 pages, but of 20 million lines of code, that's a intermediate representation. Really, we want to get back to the core DB level understanding. that's the core asset for these folks and what's your mix and then it's the property of the Antibus. Hey everybody, there's not a week that goes by when I don't get the strangest of compliments. Someone will stop me and say, Peter, you've got such nice skin. Honestly, I never thought, especially at age 64, I'd be hearing anyone say that I have great skin and honestly, I can't take any credit. I use an amazing product called One Skin, OS01 twice a day every day.

48:29The company was built by four brilliant PhD women who have identified a 10 amino acid peptide that effectively reverses the age of your skin. I love it, and like I say, I use it every day twice a day. There you have it, that's my secret. You go to 1skin .co and write Peter at checkout for a discount on the same product I use. Okay, now back to the episode. Brian, given your background in the military, I mean probably the one institution that's got the largest repository of ancient code is going to be the US government. Can you attack all of that? Unlock massive productivity. There's a lot of fear that the US is a falling empire.

49:13Its inability to understand and legislate efficiently. Couldn't you have a single massive impact on the US government? Peter, I live in Back Bay here in Boston, and so does this lead investor at Inkytel, or so he tells me because he keeps running into me when I'm walking to work and just bumping in and see how things are going. Which has to be skeptical. But the sure answer is yes. The guy's going to be sort of a fantastic end customer to ultimately modernize, to get your flights there on time, to make it in IDs easier. right? And so this is like an absolutely part of critical infrastructure that is a target customer that we have certain length.

49:54Do we have them today? No 12 months will be serving them like I think yes. Yeah I mean that could catapult you into a you know a decade billion dollar company easily just landing that kind of a customer because once you've modernized you know for one of the agencies they're all going to want it. Yeah I think we have the most top secret security clearance to patent ratios of any of any company out there. I'd like to maybe pull on the theme that we've talked about on the pot in the past, the elephant in the room, which is recursive self improvement. So how much of Blitzy is written by Blitzy? A lot.

50:34So it's interesting, right? This is actually a question of where is the core value? And I would say every single sprint that we do from a software of all my perspectives is driven by PILOTC. And so I would say a significant amount of the the corpus of the code is driven by us. Now there are algorithms that are not about writing code, right there about core invention. And so I think most of the companies, you know, Core IP is not going to be the software that it trades. It's going to be some core invention that this is around. I think about Google with PageRank, right? And I mean, Bethra Larry Page, not actual pages.

51:10But PageRank is really the core source of their original IP and sits invented a number of algorithms at the core of what see that allow us to for instance always compile code to never have circular dependencies. So you could actually rewrite sort of a core piston what see with what see pretty readily pretty quickly and it would look like it but would it do what we do the answer is sort of sort of know and it gets the question like what is the source of IP for companies and it's gotta be a breakthrough in invention. Alex, you're doing right for these large companies we're telling them for many of them, we're telling them how to use AI, right?

51:43We're coaching them and consulting with them, how do you split see? You cannot do that unless you've actually used it yourself and perfected the process, right? Because they're not just making the engineering velocity changes or the, you know, tool -user changes, they're also making the process changes. And that's why this is critical. Alex, I'd love you to take a second and dive into the Sway Bench metric here. Again, if you could, for those who are not familiar, you know, a little bit of the origin you said from Princeton and ratified by OpenAI, but what is it measuring? Who were the top of the leaderboards before?

52:15Then I want to get into the conversation about how do you compete against the Mag7 or against the frontier models in this regard? Let's contextualize it first, understanding what this wee benchmark really is. Sure. Why don't we start with the title of the white paper because that's going to drop same day that this podcast is. So people need to find the papers. And we should put a link into the white paper in the show notes here as well. Alex, please. Sure, so maybe let me just speak to SweetBench. So SweetBench is a benchmark that measures the ability for AI systems to solve typical software engineering SWE for sweet tasks in the specific form of responding and solving issues on GitHub, a very popular source code management system.

53:06So, we bench as a whole, not so we bench verified, consists of a couple thousand instances of tasks in which the central challenge that's posed to an AI is to respond to an issue in a code base. And what would happen in a normal software engineer in context is - What kind of issue, Alex? So a wide range of issues could be bugs that need to be fixed, other performance issues, and the usual workflow in a software engineering context is an issue will be identified and pull request will be submitted. So identifying an issue, responding to an issue, submitting a pull request that responds to an issue satisfying unit tests.

53:55These are all standard parts of what would be architectically considered software engineering. And so we bench, I think, is sort of an excellent industry standard at the moment that attempts to capture the life cycle of high value ad labor that a typical software engineer would perform. Now, I'll let the guys respond to their white paper. So, yeah. And so, I'll respond by asking your core question, Peter, which is, I only just want to compete with the Mag 7 in this utterly important labor task, right? And the reality is there's a significant amount of the Mag 7 at the core of what what he does.

54:38So, we use Gemini's models. We use Anthropics models. We use OpenAI's models. But the, what's unique about this technology moment in time is when you use these models against one another, the quality moves up quite dramatically. And when you use them against one another hundreds of times, right? In hundreds of different combinations, which hundreds of different tool sets and prompts, the combination goes up even more exponentially, right? And so really it's the art of orchestration through what we call extended inference time validation to move up the quality of code, ensuring at every moment in time the system has the right context to operate despite the large scale underlying code base.

55:15So we say we're excited when Gemini or OpenAI released a new model like our product gets just dramatically better. That's a really important insight, right? Building so that the better your components are, the stronger you are as a whole. And that's a unique niche. When did you realize that? I mean, that's sort of a fundamental for you. I think we realized it back when we were initially figuring out this first project together, but I'll let's sit expand on this. I think we made a bet and we said that there's not a doubt like we were building this when models had 5 ,000 tokens of context. And we made a bet that there's no doubt that context windows are going to expand and the models are going to get better at writing code.

55:56Now do we want to go and compete with the PT Max 7 and build our own model or do we want to stand in the holders of giants and use the technology to solve the problems that they're actually meant to solve. And that's exactly what we did. It's amazing. And just for context again, who had the record before you? Who had the record last week? I think there were some open source labs and there's also been some other unbublished reports that have claimed, you know, around 80%, but the 86 .8 % that we're claiming has is unprecedented in the highest number of U .S. I think by dance was the most recent trade model to be at the top there.

56:36So, you know, we can re -brand this. US, West China, if you want to. Every time I hear 80 something percent, 90 something percent, I'm thinking that these benchmarks are getting super saturated, right? And so where does this go next? I mean, what are you going to measure when you're at 100 percent? Yeah, we talk about this in the white paper. Like, this is, we certainly need new benchmarks, but the reason we didn't distribute bench fairfide for a long time is it's just not representative about the scale of problems that most people use puts in for. So the typical core quest size is like 100 lines of code.

57:11And the largest repository is a million lines within this. And so we really need a set of benchmarks that's on Linux and VS Code, which is 20 million and 4 million lines respectively, with hold outsets against those, trying to do larger scale work to ultimately show how far we can push the bounds on autonomy. Yeah, and in terms of Peter's saturation question, the paper does a really, really good job of describing the landscape of benchmarks and sweet bench in particular and the need for a new benchmark. But one of the points it makes is that when you score 86 % on this benchmark, that's effectively very close to 100%.

57:50Because the remaining subset of questions are just flawed. They're not harder. They're just not structured well. And so you've basically capped out this benchmark now. So folks, one look up the paper. What's the name of the paper? Do you have it? Yeah, Alex is urging us to retitle it. So I'll give you the subtitle, which is a domain -specific context engineering paired with extended inferncyne validation breaks the barriers of LLM and tripping software development. So that's really what we're talking about. And that's quite - It rolls off the tongue and onto the floor. Ha -ha -ha -ha. It is a technical paper.

58:31Yeah. You know, Sid, you can search Blitzy and Sid's name. He has a searchable name, Brian Elliott. There are thousands of Brian Elliott, so it turns out that Sid Parteshi plus Blitzy will get you to the paper. Okay, perfect. Alex, where do you want to take this next? What have you found important and fascinating about what Blitzy's doing? What's the implications? It's a long term buddy. It's so interesting, so many different directions to go in. I maybe just to go back to this idea of great projects, because I think Blitzi has the potential to be sort of an embodiment of an era when we just, again, turn all the AI agents loose on all of the problems in a discipline.

59:09So what I heard, I think Brian, you say a few moments ago, was something like 100X price difference per line of code or per file between Blitzi with its, you know, again, congratulations, State of the Art Performance announcement and other competing tools. When I hear you say 100x price difference, I immediately internally say, oh, well, that's just two years worth of cost hyper deflation. So you're sort of two years more expensive than the competition is the way that I heard that. So if you project forward two years, three years, four years, do you think we will find ourselves in a world where AI really has, the great refactor has been completed.

59:53And we've rewritten all of our foundational systems with Blitzy or maybe copycats of Blitzy. Do you think we find ourselves in a near future like that? I would say pontificate first here. I think we will see the models get significantly better at doing this and the cost go down. The key thing that I would like to underscore is a lot of the approach at some of the labs and then what's happening right now is The labs aren't really making money on the inference that they're running right for all the models But what we're doing is because we're using the labs and we're able to charge a premium right for the work that they see does and also provide the Validations with it.

1:00:37We're not losing money on the code that you're writing, right? So as this equation improves work period of time, the difference that Blitzy is able to create is going to also grow. So I think somewhere in that double negative, I heard the answer is that yes, as hybrid deflation kicks in, call it an order of magnitude cost reduction per year, maybe more, not only does Blitzy become very profitable, but also becomes very feasible to start to tackle these solve everything level grand challenges in software engineering. So I've been attempting to kick the tires on Blitzie myself. My first project with Blitzie was I wanted to rewrite Python, the very popular programming language.

1:01:27And I gather you, Brian, and Sid, you've had access to your own product longer than I have, which has only been two days. Have you tried to take some large scale project? I think Brian you mentioned Linux a few minutes ago. Have you tried to take some large project and either say, gosh, I want to ask Blitzie to improve performance by 10 % on some relatively mature code base or add some crazy transformative new feature. Have you tried that? Yeah, we did the fun project. We've done a number of these acts, but the one of the most fun thing we did, we onboarded VS code and we said, hey, add a chat experience to VS code.

1:02:15At the time, there was, of course, I still is one of the biggest tools out there. We tried and we built one of a subset of the features, of course, using PTC and we tried using that internally. So anytime we consider SaaS products at this point of time, We're trying to first see if we can replicate that internally using policy. If we're a few months out or a few years out, we'll say, hey, let's just use the SaaS products, the starting point and consider rebuilding it later on. I think a lot of enterprises will do this. Have you thought, I mean, so sort of free marketing advice before the public.

1:02:55Have you thought about taking all of these open source projects that are in many cases starved of core development team members and hungry for human capital or human capital equivalents, taking these projects and just aggressively setting loose the AI agents to submit very friendly, very polished pull requests to these projects to launch improvements. We've actually done that for MSL. Yeah, we've done this for one that enterprises specifically rely on. It's quite possible the best BDR, which is sending pull requests to open source libraries. And we, there's an open source one right on my homepage of our website that I think you'll find fascinating Alex, which is it goes all the way back to the beginning of the episode.

1:03:40So AWS invented this or created this repository specifically to be incredibly messy mainframe code, right? So all the different styles to represent, you know, different decades of people working on it. And ultimately say like, how would one use cogeneration to be able to move this from target code ball to target Java? We ran that through Blitzy, because the mainframe is such a big problem in all these large, even government organizations. And we've moved it from Coball to Java completely autonomously with the ability to compile right out of the box. Right. Now there's a sort of like remaining development to work on some runtime stuff.

1:04:16But this is a sort of multi -year -long project to move mainframe from a target messy Coball into Java. And the results of that have been probably one of the best business development tools that we've ever created. How fast did you do it? How long did that take? A couple days? If you've got everything from start to end, it was a week's worth difference. Ingesting. Amazing. So if we project that going, maybe if I may back to Peter's question about what the human equivalent of this is, do you have any metrics that you can point to for either cost savings relative to humans for a given unit of code, a line of code or a profile or how much faster than humans this is in general.

1:05:02We typically see from a speed perspective of 5x philosophy difference when this is brought into the enterprise and the biggest challenge is really the operational deployment. And so you're used to sort of starting your work the same day that you picked up your IDE. So what we're having these organizations do is start to sprint for the next development work the week prior. And so instead of developers starting with tickets and tasks, they're starting with code that's mostly written and a project guy with all of the human tasks to begin. So we really focus on this 5x. Anytime we engage and enterprise to work with us, we say pick a real world project and you have upcoming next quarter.

1:05:38Let us prove a 5x difference and we'll, you know, we should do the remaining development work so we have an end -to -end solution. And if we do that, then you adopt Dopplits across the org, right? And it's incredibly successful because people don't realize the cost of coordination and development and all of this sort of requirements to actually get a piece of full software out that when you can offer a significant chunk of that to agents, the velocity gains are honestly unbelievable. That's really cool insight because a lot of the younger teams are going into enterprises and then they've never been in enterprises before and they're saying, look, the raw code generation is a thousand X, 10 ,000 X.

1:06:11You know, like, yeah, but what's it going to do for me in my enterprise? And they're very few that are credible in saying, well, we actually have done it. And we know the final answer. And right now it's 5X, apparently. But they don't know like the overhead of the, you know, the organization and the documentation and they're just getting all of the things that happened before you can even start code generation. And so it's really nice actually to have at least one vendor that understands how to get the real thing. We actually need this to work in the end. It can't just be a hypothetical thousand X.

1:06:45It's three years from now. We've got digital superintelligence is landed. It's come out of, I'm going to put my bets on Google, but we'll see. What does Blitzy look like? Yeah, I think Blitzy is the core system of record and system of action for software development in the NFS environment. And so the source of truth, those from documentation, and code, which is sort of like this hybrid today, to organization for line -on -polices, hybrid graph vector database that understands a core functionality, and organizations are going to be able to move incredibly quickly from a software perspective, and the source of value is going to be core IP, that's not easy to replicate.

1:07:29To add more to that, Peter, there's always going to be some tasks where it's better to have a human in the loop and do them sequentially, right? You're solving a problem that has never been solved before and you need a quick feedback from AI, right? That's always going to be the way you use the co -pilot. But there's always going to be this other category of tasks where you can automate them away, right? Build the code, run it, deploy to production, and execute maintenance. Let's see now, you know, gives you the code. The code is the final output, but we're going to go into autonomously maintaining, deploying and keeping the applications running.

1:08:02So you're not going to need humans for specific sections of the entire enterprise. It's all going to be driven by AI. Interesting. I think Alex was about to paint a kind of a two -year view. He asked a question about what's the force multiplier today? But then I think we were going to next segue into okay, but there's 100x and another 100x coming. So I would love to finish that thought, Alex. Totally. So there's a lot of, to Dave's point and Ryan and Sid, I think you were starting to gesture in this direction as well. There's a lot of interest in the benchmark out of meter that's measuring the effective time of autonomy, the characteristic time scale over which AI systems, including AI cogen systems, can basically operate without a human intervention, sort of like a disengagement with a driverless car, how far can it drive without a human needing to take the wheel as it were?

1:08:55So I'm curious, have you thought about the characteristic time scale over which Blitzi is able to do autonomous co -gen or the human equivalent, really, of autonomous coding before which the human needs to step back into the loop and be involved right now? If I remember correctly, the current state of the art is something like one to three hours. There's a nice very clean on at least on a semi -log plot expectation that and I think we've discussed this previously if you Projected out a decade or two week we get to many many years and perhaps hundreds of millions of years and in a few decades Where does Blitzie fall in this apparent exponential trend towards exponentially increasing times without humans needing to be in the loop?

1:09:43I think this is like a project If you think about all the pieces needed to achieve this, let's take away small examples, let's take the AWS example. There's the part where you identify the requirements, decide what you need to do, get the code, and there's a part where you get all the way to production. If you look at those parts, each of them have already been automated in isolation. For example, CICD, how do you deploy the code to production? You have automations for that. Debugging, security analysis, monitoring in production, tracing and viewing the logs, ensuring that the system is not doing it, anything malicious.

1:10:16All of these items exist today and there are these blue layers that we are now seeing like for example, MCP, A to A, that allow agents to form this mesh and automate work. The only thing that's left to be done really in my opinion for these projects is to just connect the dots. And that's exactly what we're working on. So to answer your question, how far out I would say we are months out actually from delivering projects completely economically as long as they meet a certain set of criteria and conditions. So what I just heard you say said, correct me if I'm wrong is that the documentation writers, the spec writers are the new limiting factor for the field of software developments.

1:10:54Is that correct? That is correct. Great answer. How do you think about automating that process if at all? That is also automated. So if you go to Chad GPT, right, and you ask it to write documentation, it will following your criteria. The reason we have document, you know, writers in the first places to have quality, right? We have concerns that models lose context or period of time, and they skip and omit things or they can be gaimed into adding things that you don't want. We're really concerned for quality control, which is why we have humans. But as you can see, we've solved the context problem for large -code basis and there's nothing really stopping anyone for that matter from effectively adding in the right safeguards and layers of protection to ensure that we minimize any for humans, I think it's a matter of us becoming comfortable with AI doing that and I definitely see that happening over the coming months.

1:11:47To try to put it down. Begging for a follow -up white paper because Alex's question is infinitely recursive, right? If you said, okay, well then that's not the constraint then what, you know, because there's always going to be a constraint is turtles all the way down. You got to just ask him. Okay, speed up light. Speed up light. Yeah. Yeah, Douglas Adams, right that famously pointed to that the problem is far more, far harder to pose than the solution and to the extent that the new limiting factor is the spec writer or the prompt engineer or whatever we end up calling it in the future. I really would like to press you sit on this.

1:12:28Like when do we get our automated program manager, product manager, spec designer, documentation writer, if that really is the limiting factor for the speed of software engineering and the new automation all the way there. I'll tell you something Alex, you know, you know how the Blitzie tag from works, we have these thousands of agents and each of these agents has a persona. There is a product manager agent. There is a software architect agent. There is a QA agent and there is an agent that writes the prompts for the other agents. So all of the challenges that you're describing are live in production.

1:12:59With the Blitzie tag. This is a very important question though, because, you know, I know it sounds very hypothetical, but you're looking look at this timeline on Sweepench here. This is only 18 months ago that you got like 12%. And now it's saturated. It's only been 18 months. So, you know, like, what we think of is the distance scientific future, you know, it's all science fiction. It's only a year and a half in the future. Star Trek's coming buddy. Really, really hard to anticipate. Yeah. Now I love this question. I want to, I want to, I want to wrap this, wrap this, wrap this, wrap this, conversation amongst all of us on a particular topic.

1:13:39We opened up talking about trillion dollar pay packages, trillion dollar investments, you know, numbers that are extraordinary. And the sovereign funds, the venture funds, family offices are just supporting this with massive capital inflows. And so the question is that I put to all four of you, think about competing in the long term with the Mag 7 who've got this incredible access to capital. How should founders consider going about that? What's your advice to others who are getting in here during this period of exponential growth in the AI economy? How do you compete? How do you think about that?

1:14:24I think you want to be a large customer of those folks as well. We are major customers of all the, yeah, I've rented your lunch. And so they're quite, quite excited that we're going to continue to push the bounds of autonomy. And their market paths are going to continue to grow, probably dramatically, in line with that return on investment. And you know, let's see, he's going to ride those waves as well. And so if you're happy when they're successful and they're happy when you're successful, then I think you're at a pretty good strategic position. But there's one more thing, I don't do that. What I'd like to go back to a Dave said, Mark or for example, I was able to do that because it went deep.

1:15:00I think that's also the case for us. We've seen the enterprise, when I, from the enterprise, I have the challenge and the enterprise perspective and the security roadblocks and the product roadblocks and the process gaps that stop them from taking the full advantage of the product. So if you're an entrepreneur or a founder and you've seen this personally and you've struggled with this problem and you think you have a solution that addresses the core of that and you've been able to test that with the actual enterprise and demonstrate effectiveness. I think you're holding on to something that is core.

1:15:30So you're saying that you understand the problem deeply? Yes. Understandable because look, you have these Vax 7, they're giants, they have all the money and that's fine, right? You're an entrepreneur, you're an individual, you can find the right investors, we were grateful to find, you know, Dave who believed in us the moment we pitched it. And we were able to get just right amount of capital to get started. And that's really all you need. If you have the right talent, the right amount of capital, and you have the right problem that you're going after that you've convinced about because you've experienced and sold it, then you're going to be so nimble and make these moves and get a product out that is significantly better than anything that the max seven get put together because they're struggling with their own challenges like bureaucracy see and all of the hurdles that they have to go through to actually put out A .G.

1:16:16Stregling with politics. What does it say over dinner with Donald Trump? Exactly. Why don't you extract it all of that? You can build a KKAS product, get it to market, solve real world problems and you've changed the world effectively. Alex, what's your thought? How do you how do founding entrepreneurs compete with companies? I mean, I remember famously, Amazon was out there as a platform for people to sell their products. But then when Amazon saw a product that had incredibly high margin and growth, they would clone the product and compete directly. How do you keep from that happening? Two words, solve everything.

1:16:57The world is filled with so many problems that start up standing on the shoulders of the trillions of dollars of CAPEX that are being invested in cloud AI chips, fabs, energy are now poised to solve so many problems, thousands of problems. I think Brian and Sid and again, congratulations on the benchmark announcement are well poised potentially to solve the problem that we face of decades of civilizational software -croft legacy code that's just piled up without enough human capital to invest in reinventing it. And now I think we're arguably on the verge of doing that. That's one of thousands of problems, entire domains that can be solved.

1:17:44Progenial. What's solved by alpha fold, essentially overnight transforming a subset of structural biology. So many more opportunities. Before I go to you, Dave, just want to remind people, I define an entrepreneur as someone who finds a juicy problem and solves a juicy problem. And the more entrepreneurs in the world, the more problems they get solved, the better the world is. That's why we're going to hit on this over and over again. I think the career of the future is being an entrepreneur, finding a problem, falling in love with the problem, not the solution, not the tech. Because if you understand the problem deeply as the tech evolves and continues, you're going to use the newest version to go and solve that problem.

1:18:22And again, some of my favorite lines, the best way to become a billionaire is help a billion people. And the world's biggest problems are the world's biggest business opportunities. So that's what entrepreneurship means. Dave, you see hundreds and thousands of companies. You've got how many companies right now in the link studios? 28 in the building and about 50 total. Amazing. What are you when you're looking to invest in a young entrepreneurial team like Brian and Sid or like the founders of Mercor or again, some of the incredible unicorns that that we've backed out of link, exponential ventures, what are you looking for to make sure that that company isn't gonna get disrupted in the wake of an open AI or Google slight jog to the right?

1:19:15You know, it's funny, Kevin Wheel, we asked that exact question in that podcast we did two weeks ago, and he answered exactly the way I had hoped he would answer, which is, in a world where the foundation model companies is get to AGI and can do virtually anything. Are you just gonna take over the world? And, you know, Kevin was really clear that maybe we can do that maybe we can't, we probably can't anyway, but even if we could, we don't want antitrust to come in here and break us up. You know, that's the last, we want a huge thriving ecosystem of partners that give us money. You know, it's Blitzy one of those companies that gives us money.

1:19:50Yes, therefore there are best friends. Go conquer the world. I love that. Take over, change the entire foundation of all legacy codebase, make a trillion dollars and give us half of it. They're all be happy. That's what they want. I took a two hour walk yesterday with a dear friend of mine here who runs a large venture fund. And we're talking about the notion that his bet was, you know, Google had so much more capability than they unleashed. And they said, look, it's an open AI go and do as much of this as you can because we need someone out there competing with us. otherwise we'll get broken up for antitrust regions, which is a fascinating idea.

1:20:26You need viable competition to help you price to help you remain on the edge, to help you not be sort of broken down by the government. And be a good partner. When Google was growing like crazy and we had all these portfolio companies, we made a ton of gains, but be a good partner to Google while they're growing like crazy. And now it's the foundation. Just be a good partner. talk to them all the time, make sure you know where they're going and they'll love you. Amazing. I have a selfish question. I don't know if we're running out of time, Mary, but... No, it's fine. It's close with yourself as question.

1:21:01Okay, okay. Well, this time I'm always looking for traits. Like this has obviously been one of our best investments ever. And this guy is the limit from here. And I'm always looking for traits of success. And the morale at Blitzy is like nothing I've ever seen. You know, which is not a no -brainer. When you're doing video generation for a movie studio or whatever, It's easy to keep in Hymer Al, but when you're doing, you know, five million lines of code core cobalt conversion But yet you guys have just this crazy thriving culture and Sid mentioned, you know, we're we're first money in I don't remember why we love the deal so much I do remember we absolutely was a no -brainer to invest in you guys So two things jump out at me one of them is bits Which is just the hardest place in the world to get into and in video, which is you know, you've seen growth The other one is Brian.

1:21:43I think you had army ranger Bangalore Institute of Technology That's the Billah Billah Citroescheknitz. Okay. Yeah. It's a cool name though, bits. It's like MIT, BIT, but it's bits. And it was, you know, it was, by the way, MIT designed the curriculum for bits. So that statement, that was actually true. Oh, that's cool. The other one though was, you know, Brian, I think you had, not just West Point, but Ranger Training, which is freakishly hard. And then first boots on the ground in Syria. So literally the first people touching a war zone. So I got to feel like there's something in those experiences that puts you a cut above in terms of building a team, managing logistics, building morale.

1:22:28Any clues there that other founders can pick up on and... Yeah, I think some evidence of being incredibly mission driven and ambitious is what you would see if you were anthropologists looking at both of our backgrounds. But if you take me for instance and if you fast forward or I guess rewind to 2017 when I was serving in the 75th Ranger regiment the the mandate was a go into Syria There's about 2000 ice ins fighters in that hold rocker We're gonna send you with a hundred guys recruit everybody else and take back the city, right? And oh by the way, we can't let anybody in the United States know we're here because we're the co -vertly right and And to be able to sort of sort of go in and solve that problem like that's a very ambitious undertaking where work like conquering cities These isn't something that most people have spent their time doing.

1:23:14So when you look at the level of ambition of the company, everybody here at the business of what's you, has that ethos. And the very first thing we do in our interview is we screen for ambition and the ability to invent and make. We have those four values. And if you talk to any single person that says in this building right here, they will tell you and the right that what we're doing is one of the most important things they do in their lifetime because the economic expansion that the glow gets, the GDP expansion that you get from automating software development or at least huge chunks of software development, there's almost no better incremental use of energy than driving towards that goal.

1:23:55Wow, that's a beautiful title. Are you guys in 996 or 997 shop? It's Saturday day, we'll be honest. We're in September, we're 997 categories. Oh my god. Just a back day about one is on his nine nine seven. I was in a state. I hope my dance was the last leader on Sri Vengevera fight. So we can work longer than them and beat them on leaderboards. Oh my God. Guys, listen, congratulations on hitting that new benchmark. But more importantly, thank you for the work that you're doing from the companies that will benefit from our government that will benefit from the world that will benefit. But this is your upgrading the DNA of industries and of our planet.

1:24:39So grateful for you. Alex, Dave, any closing thoughts here? I'm just super excited to see what you guys can bring to the future. I would very few things would excite me more on the software engineering front than a few years from now to learn that the entire software stack that I run, that companies that I work with run has been 99 % rewritten, bi -blitcy, bi -blitcy's agents to remove all the vulnerabilities, improve all the performance. I think it's the sort of challenge before you guys that sets us on the road to recursive self -improvement and abundance and also solving everything in software.

1:25:26Solving everything. That's my phrase for the day. Let's solve everything. It's better. It's better to Dave's 997 than 10 -peng. 10 -peng the opposite of 996 line flat in response to overwork. Dave, let's give my closing thought. Definitely everybody read the white paper. The title may sound very complex, but the paper for itself is very, very readable. So please read it and then your takeaway will be, wow, okay, now we need a new benchmark. Inside Baseball, Blitzie's already working with MIT to create the next generation benchmark. So, but catch up to what they did right here by reading the paper.

1:26:04To our subscribers, thank you for following Moonshots and WTF episodes. We're grateful for your time. We hope that in spending the time with us, you're able to understand how incredibly powerful this technology is for transforming our world, our lives, creating a future of abundance. I hope this counters all the dystopian news you get on the six and seven o 'clock news, that stuff I don't watch. This is the stuff I focus on. I hope you do too. I'm grateful to my moonshot partners, AWG, Dave Blunden, Selene, wherever you are, transiting the Atlantic to come back here to the US. And again, Bryant, and Sid, congratulations on your epic wins, excited for your future success.

1:26:44Every week, my team and I study the top 10 technology metatrends that will transform industries over the decade ahead. I cover trends ranging from human under robotics, AGI, and quantum computing to transport energy, longevity and more. There's no fluff. Only the most important stuff that matters, that impacts our lives, our companies, and our careers. If you want me to share these metatrends with you, I writing newsletter twice a week, sending it out is a short two minute read via email. And if you want to discover the most important metatrends 10 years before anyone else, this reports for you.

1:27:16Readers include founders and CEOs from the world's most disruptive companies and entrepreneurs building the world's most disruptive tech. It's not for you if you don't want to be informed about what's coming, why it matters, and how you can benefit from it. To subscribe for free, go to demandis .com slash metatrends. to gain access to the trends 10 years before anyone else. Alright, now back to this episode.