Cline: the open source coding agent that doesn't cut costs

27:09It's basically like you can do a HGCP request that includes payment in it. What? Yeah, it's been around forever, the 402 error that's like payment not accepted or something, right? So yeah, we've seen some people talking about that, more natively building that in. But yeah, no one's really doing that right now. Anything you're seeing on, are people making MCP startups that are interesting? mostly around re-hosting local ones and do remote and then basically do instead of setting up 10 MCPs you have like a canonical URL that you put in all of your tools and then expose all the tools from all the servers yeah there's like MCP that run some of these tools yeah but I think it kind of has the same issues of how do you incentivize people to make better MCPs you know and will it be mostly first party or will it be third party yeah like your perplexity MCP was the 4-2 what was wrong with the perplexity one With MCPs and installing them locally on your device, there's always a massive risk associated with that.

28:15And when an MCP is created by someone that we have no idea who they are, at any point they might update the GitHub to introduce some kind of malicious stuff. So even if you verified it when you were listing it, you might change it. So I ended up having to fork a few of those to make sure that we lock that version down. Oh, okay. So this is just like you're just forking it so that you don't change it. without that. Interesting. These are all the problems of a registry, right? Like that you need to ensure security and all that. Cool. I'm happy to move on. I would say like the last thing that's kind of curious is like if Anthropic hadn't come along and made MCP, what would have happened?

28:53What's the alternative history? Like would you have come with MCP? So we saw some of our competitors who have been kind of working on their own version of plug and play tools into these agents. They kind of had to natively create these tools and integrations themselves directly into their product. And so I think anybody in the space would have had to just do the laborious work of having to recreate these tools and integrations. So I think Anthropic just saved us all a lot of trouble and tapped into the power of open source and community-driven development and allowed individual contributors to make an MCP for anything people could think of and really take advantage of people's imagination in a way that I think is necessary right now for us to really tap into full potential of this sort of thing.

29:39We've had, I think, a dozen episodes with different coding products. And by the way, this episode came directly after he tweeted about the Cloud Code episode, where they were sitting right where you're sitting. Thanks for sharing the time about RAG. Can you give people maybe the matrix of the market? You have fully agentic, no IDE. You have agentic plus IDE, which is kind of yours. You have IDE with some co-piloting. How should people think about the different tools and what you guys are best at or maybe what you don't think you're best at? I think what we're best at and like our ethos since the beginning is just meet the developers where they're at today.

30:17I think there is a little bit of insight and hand-holding these models need right now. And the IDE is sort of the perfect conduit for something like that. You can see the edits it's making. You can see the commands that it's running. You can see the tools that it's calling. It gives you the perfect UX for you to have the level of insight and control and be able to course correct the way that you need to to work with limitations of these models today. But I think it's pretty obvious that as the models get better, you'll be doing less and less of that and more and more of the initial planning and prompting and sort of have the trust and confidence that the model will be able to get the job done pretty much exactly how you want it to.

30:55I think there will always be a little bit of a gap in that these models will never be able to read our minds. So there will have to be a little bit of, you know, making sure that you give it the most comprehensive and sort of like all the details of what you want from it. So if you're a lazy prompter, you can expect a ton of friction and back and forth before you really get what you want. But I think we're all learning for ourselves as we work with these things, kind of the right way to prompt these things and to be explicit about what it is that we want and kind of how they hallucinate the gaps that they might need to fill to get to the end result and how we might want to avoid something like that.

31:32So what's interesting about Cloud Code is there isn't really a lot of insight into what the agent's doing. It kind of gives you this checklist of what it's doing holistically at a high level. I don't think that really would have worked well if the models weren't good enough to actually produce work that people were generally happy with. We're kind of there. And I think the space has to catch up to, OK, maybe people don't need as much insight into these sorts of things anymore. And they are OK with letting an agent kind of get the job done. And really, all you need to see is sort of the end result and tweak it a little bit before it's really perfect.

32:05And I think there is going to be different tools for different jobs. I think something like totally autonomous agent that you don't have a lot of insight into is great for maybe scaffolding new projects, but for kind of the serious, more complex sorts of things where, you know, you do need a certain level of insight or you do need to kind of have like more engagement. You might want to use something that does give you some more insight. So I think these sorts of tools complement each other. So for example, writing tests or spinning off 10 agents to try to fix the same bug might be useful for a tool that doesn't require too much engagement from you.

32:43Whereas something that requires a little bit more creativity or imagination or extracting context from your brain requires a little bit more of insight into what the model is doing and a back and forth that I think client is a little better suited for. like visibility into what the agent is doing. That's like one axis. And then another is autonomy, like how automated it is. And we have a category of companies that are focusing more on the use case of people that don't even want to look at code, which is like, you know, the lovables, the replets, where it's like you go in, you build an app, you might not even be technical, and you're just happy with the result.

33:22And then you have kind of stuff that's kind of like a hybrid where it's, you know, for engineers, it's built for engineers, but you don't really have a lot of visibility into what's going on in the hood. This is like for like the vibe coders, where they're, you know, fully, you know, letting, letting the AI take the wheel and building stuff very rapidly. And lots of open source fans and, you know, people that are hobbyists enjoy coding in this, in this manner. And it is really fun. And then you get to like serious engineering teams where they can't really give everything over to the AI, at least not yet.

33:57And they need to have high visibility into what's going on every step of the way and make sure that they actually understand what's happening with their code. You're kind of handing off your production code base to this non-deterministic system and then hoping that you catch it in review if anything goes wrong. Whereas personally, the way I use AI, the way I use Klein is I like to be there every step of the way and kind of guide it in the right direction. So I know every step of the way, like as every file is being edited, I prove every single thing and make sure that things are going in the right direction.

34:32And I have a good understanding as things are being developed where it's going. So like this kind of hybrid workflow really works for me personally. But, you know, sometimes if I want to go full YOLO mode, I go ahead and just auto-approve everything and just step out for a cup of coffee and then come back and, you know, review the work. My issue with this as an engineer myself is that we all want to believe that we work on the complex things. How have you guys seen the line of complex change over time? I mean, if we sat down having this discussion 12 months ago, complex was like much easier than today for the models.

35:09Do you feel like that's evolving quickly enough that like, you know, in 18 months, it's like you should probably just do full of GenTech for like 75 % of work, 80 % of work? Or do you feel like it's not moving as quickly as you thought? I think what was complex a couple years ago is totally different to what is complex today. Now, I think what we need to be more intentional about are the architectural decisions we make really early on and how the model kind of builds on top of that. If you have kind of a clear direction of where things are headed and what you want, you kind of have a good idea about how you might want to lay the foundation for the code base that you're producing.

35:46And I think what we might have considered complex a few years ago, algorithmic challenges, that's pretty trivial for models today and stuff that we don't really necessarily have to think too much about anymore. We kind of give it a certain expectation or unit test about what we want and it kind of goes off and puts together the perfect solution. So I think there's a lot more thought that has to go into tasteful architectural decisions that really comes down to you having experience with what works and what doesn't work, having a clearer idea for the direction of where you want to take the project and sort of your vision for the code base.

36:20Those are all decisions that I think is hard to rely on a model for because of its limited context and its inability to kind of see your vision for things and really have a good understanding of what you're trying to accomplish without you putting together a massive prompt of everything that you want from it. I think what we spent most of our time working on a couple years ago has totally changed. And I think for the better, I think architectural decisions are a lot more fun to think about than putting together algorithms. It kind of frees up the senior software engineers to think more architecturally.

36:54And then once they have a really good understanding of what the current state of the repository is, what the current state of the architecture is, and when they're introducing something new, they're really thinking at an architectural level. And they articulate that to Klein. And that's also, there's like some skill involved there. and some of that can be mitigated with like asking follow-up questions being proactive about clarifying things on the agent side but ultimately you need to articulate this new architecture to the agent and then the agent can go down and down into the mines and implement everything for you and it is more fun working that way like personally like i find it a lot more engaging to just think on a more architectural level and for junior engineers it's a really good paradigm to learn about the code base.

37:40It's kind of like having a senior engineer in your back pocket where you're asking Klein, like, hey, can you explain the repository for me if I wanted to implement something like this? What files would I look at? How does this work? It's great for that as well. If you're moving on from competition, I have one last question. Competition. So there's Twitter beef with Rucode. I just want to know what the backstory is. Because you tweeted yesterday, somebody asked Ruko to add Gemini CLI support and then you guys responded just copy it from us again and they said thank you we'll make sure to give credit is it a real beef?

38:14no is it a friendly beef? I think we're all just having fun on the timeline there's a lot of forks there's like 6 ,000 forks yeah there's like if you search Klein in the VS Code marketplace it's like the entire page just like forks of Klein and there's like even forks of forks that came out and raised a whole bunch of money. What? Yeah, it's crazy. The top three apps on OpenRouter are all Klein and then Klein Fork, Klein Fork. It's funny. Yeah, billions of tokens getting sent through all these forks. There's like Fork Wars and 10 ,000 forks and all you need is a knife. No, it's exciting. I think they're all really cool people.

38:57We've got people in Europe forking us. We've got people in China making a little fork of us. I think Samsung recently came out with a Wall Street Journal article where they're using Klein, but they're using their own little fork of Klein that's kind of isolated. We encourage it. Do you have any regrets about being open source? Not at all. I think Klein started off as this really good foundation for what a coding agent looks like. And people just had a lot of their own really interesting ideas and spinoffs and concepts about what they thought that they wanted to build on top of it was. And just being able to see that and see the excitement around just in the space in general has just been, I think, inspirational and has helped us kind of glean insights into what works and what doesn't work and incorporate that into our own product.

39:41And for the most part, I think for the Samsung's and all the organizations where there's a lot of friction and being able to use software like this on their code bases, it reduces that barrier to entry, which I think is incredibly important when you want to get your feet wet with this whole new agentic coding paradigm that's going to completely upend the way that we've written software for decades. So in the grand scheme of things, I think it's a net positive for the world and for the space. So no regrets. In a lot of ways, it's us and the forks. We were kind of there originally when we were the only ones with this philosophy of keeping things simple, keeping things down to the model, letting the model do everything, not cutting on, not trying to make money off of inference, going context heavy, reading files into context very aggressively.

40:30and kind of going back to cloud code, I was actually like, it was really nice to see that they came out and they validated our whole philosophy of like keeping things as simple as possible. And that kind of goes in with like the whole RAG thing, which is like RAG was this early thing in like 2022. You started getting these vector database companies, context windows were very small. This was like a way of people called it like, oh, you can give your AI infinite memory. It's not really that, But that was like the marketing that was sold to the venture backers that were like investing in all these companies.

41:05And it became this narrative that really stuck around. And like even now, like we get like potential like, you know, enterprise perspective, like they're going through like the procurement process. And it's almost like they're going through like a checklist asking like, hey, do you guys do like indexing like of the code base and doing RAG? And I'm like, well, why? Like, why are you like, why do you want to do this? I think Boris said it very well on this exact podcast where we tried RAG and it doesn't really work very well, especially for coding. The way RAG works is you have to chunk all these files across your entire repository and chop them up into small little pieces and then throw them into this hyper-dimensional vector space and then pull out these random chugs when you're searching for relevant code snippets.

41:53And it's like, fundamentally, it's like so schizo. And like, I think it actually distracts the model and you get worse performance than just doing what like a senior software engineer does when they first they're introduced to a new repository, where it's like that you look at the folder structure, you look through the files. Oh, this file imports from this other file. Let's go take a look at that. And you kind of agentically explore the repository. That's like, we found that works so much better. And there's like similar things where it's like, like the simplicity always wins. like this bitter lesson where fast apply is another example.

42:26So Cursor came out with this fast apply, like they call the instant apply back in July of 2024, where the idea was models at the time were not very good at editing files. And the way editing files works in kind of the context of an agent is you have a search block and then a replace block where you have to like match the search block exactly to what you're trying to replace. And then a replace block just swaps that out. And at the time models were not very good. It was like, I forget, like GPT they were using under the hood at the time wasn't very good at formulating these search blocks perfectly and it would fail oftentimes.

42:59So they came up with this clever workaround to fine tune this fast apply model where they let these frontier models at the time, they let them be vague. They let them output those like lazy code snippets that we're all very familiar with, where it's like rest of the file here, like rest of the imports here. And then fed that into this fine-tuned fast supply model that was like probably like a Quen 7B or something quantized, very small, dinky little model. And they fed this lazy code snippet into this smaller model. And the smaller model we fine-tuned to output the entire file with the code changes applied.

43:37and that you know the one of the founders of ador said this really well in like very early github discussions where he said like well now instead of worrying about one model messing things up now you have to worry about two models messing things up and what's worse is the other model that you're giving that you're handing your production code to this like fast apply model it's like it's a tiny model, its reasoning is not very good. It's maximum output tokens. You know, there might be 8 ,000 tokens, 16 ,000 tokens. Now they're training like 32 ,000 tokens, maybe. And a lot of the coding files, like we have a file in our repository that's like 42 ,000 tokens long.

44:17And that's longer than the maximum token output length of one of these smaller fast supply models. So what do you do then? Then you have to build workarounds around that. Then you have to build all this infrastructure to like pass things off and then it's making mistakes. It's like very subtle mistakes too, where it's like, it looks like it's working, but it's not actually what the original Frontier model suggested. And it's like slightly different. And it introduces like all of these subtle bugs into your code. And what we're starting to see is like, as AI gets better, the application layer is reducing.

44:49You're not going to need all these clever workarounds. You're not going to have to maintain these systems. So it's really liberating to not be bogged down with RAG or with FastApply and just focus on this core agentic loop and maximizing diff edit failures. Like in our own internal benchmarks, CloudSonna 4 recently hit a sub 5 % or like around actually 4 % diff edit failure rate. When FastApply came out, that was way higher. That was like in the 20s and the 30s. Now we're down to 4%, right? And in six months. How does it go to zero? Well, it's going to zero. Like as we speak, it's going to zero every day, you know, and I was actually talking with the founders of some of these companies that do fast supply.

45:33They were trying to kind of work with us. Their whole bread and butter is fine tuning these fast supply models and, you know, like Relace and Worf. And I had like a very candid conversation with these guys where I was like, well, there's a window of time where fast supply was relevant. Cursor started this window of time back in July. How much time do you think we have left until they're no longer relevant? Do you think it's an infinite time window? They're like, no, it's definitely finite. Like this era of fast apply models is definitely coming to an end. And I was like, well, how long do you guys think?

46:02They were like, maybe three months, maybe less. So I still think there's some cases where RAG is useful. You know, if you have a lot of human readable documents, a large knowledge base of documents where you don't really care about like inherent logic within them, like sure, index it, chunk toolkit, do retrieval on it. Or FastApply is like maybe if your organization, you're forced into using like a very small model that's not very good at search and replace, like a deep seek or something, you know, maybe use a FastApply model. I think RAG and FastApply were these just tools in a toolkit for when models weren't the greatest at large context or search and replace diff editing.

46:45but now they are extra ingredients that could make things go wrong that you just don't need anymore there was an interesting article from the cognition labs about you know multi-agent orchestration and getting right into it it's like you're on you're on autopilot for us that's cool yeah i mean it's a great article yeah it's great they talked about how you know when you start working with different models different agents there's a lot that gets lost and the details. And, you know, the double or in the details, those are the most important things and making sure that it doesn't, you don't have the agent sort of like running in loops and running to the same issues again and have sort of like all the right context.

47:25And so I think being close to the model, throwing all the context you need at it, not taking these cost optimized approach to pulling in relevant context, use something like RAG or a cheaper model to apply edits to a file. I think ultimately, yes, it's more expensive asking a model like Claude Sonnet to do all these sorts of things, to grep an entire code base and to fill up its entire context. But you kind of get what you pay for. And I think that's been another benefit of being open source is that our developers, they can peek under the kimono. They can see where their requests are being sent, what prompts are going into these things.

48:03and that creates a certain level of trust where when they spend$10,$20,$100 a day, they know where their data is being sent, what model is being sent to, what prompts are going into these things. And so they get comfortable with the idea of spending that much money, get the job done. It's like not making money off of inference. I think the incentives are so relevant in this discussion because if you're incentivized, if you're charging$20 per month and you're trying to make money on that, you're going to be offloading all kinds of important work to smaller models or optimizing for cost with RAG, like retrieval with RAG, not reading the entire file, but maybe reading like a small snippet of it.

48:46Whereas if you're not making money off inference and you're just going direct, users can bring their own API keys, well, then all of a sudden you're not incentivized to cut down on cost. You're actually incentivized just to build the best possible agent. We're starting to see this trend of the whole industry is moving in that direction, right? We're starting to see everyone open up to pay-as-you-go models or pay directly for inference. And I think that is the future. What's the client pricing business model? Right now, it's bringing an API key. Essentially, just whatever pre-commitment you might have to whatever inference provider, whatever model you think works best for your type of work.

49:27you just plug in your Anthropic or OpenAI or OpenRouter, whatever it is, API key into Klein, and it connects directly to whatever model you select. And I think that level of transparency, that level of we're building the best product, we're not focused on sort of capturing margin on the price obfuscation and clever tricks and model orchestration to keep costs low for us and optimize for higher profits. I think that's put us in this unique position to really push these models to their full potential. And I think that's shown. I think you get what you pay for, throw a task and client, and it gets expensive.

50:10That's the cost of intelligence, right? It's the cost of intelligence, yeah. So yeah, the business model right now is you get to choose kind of where it's open source. You can fork it. You can choose where your data gets. You can choose who you want to pay. A lot of organizations we've talked to get a certain level of volume-based discounts with these providers. And so they can take advantage of that through client, which is helpful because client can get pretty expensive. And yeah. Wait, so I mean, I'm still not hearing how you make money. Like you said you don't. Huh? Why? Why make money? Yeah.

50:42Because you have to pay your salaries. No, that's the, a lot of people ask us that and I always just throw the why at them, but it's. These sound like the party fool guys. Party fool is like. The real answer is enterprise. So. Which we can say because we release this when you launch it. Yeah, so you want to talk about enterprise? Yeah, I think being open source and your API key has given us a lot of easy adoption in these organizations where things like data privacy and control and security are top of mind. And it's hard to commit to sending their code in plain text to God knows what servers, training their data on models that might output their IP to random users.

51:26I think people are a lot more conscious about where their data is getting sent and what's being used to it. And so it's given us this opportunity to say, okay, nothing passes through our own servers. You have total control over the entire application where your data gets sent. And that's given organizations that, you know, we've been talking to over the course of the last couple of months, this sort of like easy adoption. And I think this opportunity for us to work more closely with them and say, you know, what are all the things that we can do to help with adoption in the rest of your organization?

51:56Essentially, how can we pour gasoline on sort of the evangelism that people have for Klein in these organizations and spread the usage of Agenta coding, I think, at an enterprise level? Well, yeah, what's crazy is, so we had, we open-source Klein. People really liked it. Developers were using it within their organizations. Their organizations were kind of like reluctantly okay with it because they saw like we're open-source and we're not sending their data anywhere. They could use their existing API keys. And then we launched on our website a contact form for enterprise. If you're interested in an enterprise offering, hit us up.

52:34And we had no real enterprise product at the time. And it turned out like we just got this massive influx of big enterprises reaching out to us. And we had a Fortune 5 company come up to us and they were like, hey, we have hundreds of engineers using Klein within our organization and this is a massive problem for us. This is like a fire that we need to put out because we have no idea what API keys they're using, how much they're spending, where they're sending their data. Please just let us give you money to make an enterprise product. So the product kind of just evolved out of that. Right, right.

53:11I mean, it really just comes down to more of listening to our users. So right after we put out this page, we just had a lot of demand for sort of like the table stake. enterprise features, the security guardrails and governance and insights that sort of like the admins in these organizations need to reliably use something like Klein. Yeah, we've gotten a lot of people wanting us to sort of give them two things, invoices just to help with like all the budgeting and spending the thousands of dollars. All the Europeans. Yeah, just the other thing which I thought was a little bit surprising was some level of insight into the benefit that clients providing them.

53:50So it could be our sage or lines of code written because it allows these sort of like AI forward drivers for adopting these sorts of tools and these organizations to take that as a proof point and go to the rest of their teams and say, this is how much client's helping me. You need to start adopting this so we can keep up with the rest of the industry. This is for like internal champions to prove the ROI? Exactly. Okay. Use as sort of evidence for this, you know, to justify the spend. But also to promote the product in these organizations. We can do this afterwards, but we'd like to talk to those and actually feature some of them, what they're saying to their bosses on the podcast so that we can get a sense.

54:27Because oftentimes we hear, we only talk to founders and builders of the dev tool, but not the end consumer. And actually we want to hear from them, right? About how they're thinking about it, what they need. Kind of cool. One thing I wanted to ask to double click on is the relationship between Open Router and then your enterprise offering, right? So my understanding is currently everything runs through OpenRouter. Not everything. So you can bring API keys to OpenAI, Anthropic, Bedrock. And then you have a direct connection there. The user has a direct connection there. Yeah, correct. But everything else would run through OpenRouter.

55:02And so basically the enterprise version of Klein would be you have your own OpenRouter that you would provide visibility and control to that enterprise. Yeah, that's for the self-hosted option, right? There's a lot of enterprises where they're okay with not self-hosting, but as long as they're using their own Bedrock API keys and stuff like that. Whereas the ones that are really interested in self-hosting or that want to be able to manage their teams, there would be this internal router going on. The curious thing here is, what if model costs just go to zero? Gemini code just comes out and it's like, yeah, guys, it's free.

55:40Well, yeah, it'd be great for us. So our thesis is inference is not the business. You would just never make money on inference. Yeah, we want to give the end user total transparency into price, which I think is incredibly important to even get comfortable with the idea of spending as much money as you do. I think the price obfuscation in this space has given developers this reluctance to opt into usage-based plans. And we're seeing a lot of people kind of converge on this concept of, maybe have a base plan just to use the product, but sort of get out of the way of the inference and respect the end developer enough to give them the level of insight into not just the cost, but the models being used, and give them more confidence in spending however much it takes to get the work done.

56:25I think you can use tricks like RAG and Fast Apply and things like that to keep costs low, but for the most part, there's enough ROI on coding agents where people are willing to spend money to get the job done. And for a truly good coding agent, the ROI is almost hard to even calculate because there's so many things that I would have never even bothered doing. But now I have Klein and I could just do this weird experiment or do this side project or fix this random bug that I would have never even thought about. So how do you measure that? One variant of this problem, we're about to move on to context engineering and memory and all the other stuff.

57:06One variant of this I wanted to touch on a little bit was just background agents and multi-agents. So the instantiations of this now, I would say our background agents would be codecs, for example, like spinning up one PR per minute or devin or cognition. So would you ever go there? That's one concrete question I can ask you. Would there be client on the server, whatever? And then the other version is still on the laptop, but more sort of parallel agents. The Kanban is currently very hyped right now. People are making Kanban interfaces for Cursor and also for Cloud Code. Anything in the parallel or background side of things?

57:48We're releasing a CLI version of Klein. And using the CLI version of Klein, it's fully modular. So you can ask Klein to run the CLI to spin up more clients. or you could run client in some kind of cloud process in a GitHub action, whatever you want. So the CLI is really the form factor for these kind of fully autonomous agents. And it's also nice to be able to tap into an existing client CLI running on your computer and be able to like take over and steer it in the right direction. So that's also possible. But what do you think, Sad? I don't think it's an either or. I think all these different modalities complement each other really well.

58:29So the Codex, the Devins, Cursor's background agent, I think they all sort of accomplished the same thing. If we were to come out with our own version of it, I'd say that it would be a foundation for how other developers could build on top of it. So Nick's older brother, Andre, he's sort of thinking 10 years ahead. And it always kind of blows my mind a little bit about some of the ideas that he has about where the space is going. But we recently had a discussion about building this open source framework for coding agents for any sort of platform. Building the SDK and the tool necessary to bring Klein to, you know, Chroma as an extension, to the CLI, to JetBrains, to Jupyter Notebooks, to your smart car, whatever it is.

59:16But to build the... Your fridge. Your fridge, exactly. To put to... The microwave, maybe? Yeah, exactly. I mean, this is what we saw kind of like with the 6 ,000 forks on top of Klein is we sort of like put together this foundation for how this community of developers, we sort of put together this foundation that this community of developers could like build on top of and sort of take advantage of, you know, their experiments and imagination and their creativity about where the space is headed. And I think looking forward, building an open source foundation and the building blocks for how we bring something like Klein to things that go outside the scope of software development or, you know, VS Code extension.

59:53I think that'll open up the door to things that ultimately complement each other really well, but it'll never be sort of this either-or thing. I think background agents are good for certain kinds of work, and parallel can be multi-agents might be good for when you want to experiment and iterate on five different versions of how a landing page might look. And then something like a back-and-forth with a single agent like Klein works really well for when you want to pull context and put together a really complicated plan for a really complex task. And I think all these different tools will ultimately end up complementing each other and people will kind of develop a taste and an understanding for what works best or what kind of work.

1:00:30But I think just looking 10 years ahead, we at the very least want to sort of be at the frontier of providing sort of the building blocks for what the next thing is after background agents or, you know, multi-agents. I was going to go into context engineering, kind of like topic du jour. I think that this is kind of similar-ish in a thread to RAG and how RAG is a mind virus, which I love, by the way, the way that you phrased it. Yeah, you have in your docs context management. You also have a section on memory bank, which is kind of cool. I think a lot of people are trying to figure out memory.

1:01:01Let's just start at the high level and then we'll go into memory later. What does context engineering mean to you? Context engineering mean to me? It means prompt engineering. Yeah, right. So I think there is a lot of art to what goes in there. I think that really is the 80-20 of building a really good agent. It's figuring out what goes into the context. And I think interplay between MCP and your system client recommended prompts, I think is what is ultimately making a good agent. Yeah, I think context management is like, one part of it is what you load in to context. The other part of it is how do you clean things up when you're reaching the context window, right?

1:01:50How do you curate that whole life cycle from zero to maximum context window? and the way that I think about it is there's so many options on the table and there's so many risks to misdirecting the agents or distracting the agents. There's ideas about, you know, rag or other kinds of forms of retrieval. That's one idea. There's the agentic exploration. That's another idea that we found works much better. And it seems like the trend is generally for loading things into context, it's giving the model the tools that it can use to pull things into context, letting the model decide what exactly to pull into context, as well as some hints along the way, kind of like a map of what's going on, like ASTs, abstract syntax trees, potentially what tabs they have open in VS Code.

1:02:45That was actually in our internal kind of benchmarking that turned out to work very, very well. It's almost like it's reading your mind when you have a few tabs open. It stresses me out because sometimes then I'm like, I have unrelated tabs open and I have to go close them before I kick off the thing. I wouldn't think too much about it, especially when you're using Klein. Klein does a pretty good job of just navigating that. But definitely there are edge cases, right? There's edge cases for everything. And it's kind of like, okay, what's the majority use case is when are you starting a brand new task and you don't have a single tab open that's relevant to it?

1:03:17Obviously in the CLI, you don't have that little indicator. So there you have to think outside the box for that. So that's like for reading things into context. And then for context management is when you're approaching the full capacity of the context window is how do you condense that? And we've played around with this kind of naive truncation very early on where we just like throw out the first half of the conversation. It's common. And there's problems with that, obviously, because it's like kind of like you're halfway through a book and you're like you start reading halfway through right you don't know anything that happened beforehand and we like to think a lot about like narrative integrity is like every task inclined is kind of like a story it might be a boring story where it's like this lonely coding agent that's just you know determined to help you solve you know whatever it is like the child like the the big thing that the protagonist needs to overcome is like the resolution of the task, right?

1:04:18But how do we maintain that narrative integrity where every step of the way, the agent can kind of predict the next token, which is like predict the next part of the story to reach that conclusion. So we played around with things like cleaning up duplicate file reads. That works pretty well. But ultimately, this is another case where it's like, well, what if you just give the model, like, what if you just ask the model, like, what do you think belongs in context. Another form of this is summarization, which is like, hey, summarize all the relevant details and then we'll swap that in. That works really, really well.

1:04:50Yeah. Double clicking on the AST mention, that's very verbose. When do you use that? Right now it's a tool. The way that it works is when Klein is doing sort of the agentic exploration of trying to pull in relevant context and it wants to sort of get an idea of what's going on in a certain directory, for example, there's a tool that lets it pull in all the sort of language from a directory. So it could be the names of classes, the names of functions, and that gives it some idea of, okay, here's what's going on in this folder. And if it seems relevant to whatever the task is trying to accomplishes, then it sort of like zooms in and starts to actually read those entire files into context.

1:05:32So it's essentially a way to help it kind of figure out how to navigate through large code bases. Yeah, we've seen some companies working on, it's like an interesting idea. It's like an AST, but it's also a knowledge graph. And you can run these discrete deterministic, almost like actions on this knowledge graph, where you could say like, hey, find me all the functions that find me all the functions of the code base and find me all the functions that aren't being used and delete all of them. and the agent can kind of reason in this almost like SQL-like language, working with this knowledge graph to do these kinds of global operations.

1:06:07Like right now, if you ask a coding agent to go through and remove all unused functions or do like some kind of large refactoring work, in some cases it might work, but very oftentimes it's just going to struggle a lot, burn a lot of tokens and fail ultimately. Whereas with these kinds of tools, it can actually operate on the entire repository with these kinds of query, like short little query statements. I think there is a lot of potential in something like this, where it's like the next level beyond the AST and it's like a language for querying this kind of knowledge graph. But like we've seen with like the Cloud4 release is these frontier model shops, they tend to train on their own application layer.

1:06:52And you might come up with like a very clever tool that in theory would work really well, but then it doesn't work well with Cloud4 because Cloud4 is trained to grep, right? So that's another interesting phenomenon where it's like you're expecting these frontier models to become more generalized over time, but instead they're becoming more specialized and you have to like support these different model families. Just to wrap on the memory side, memory is almost the artifact of summarization. So you summarize the context and then you kind of extract some sides. Any interesting learnings from there?

1:07:27Like things that are maybe not as intuitive, especially for code. I think people grasp the memory about humans, but what are memories about code bases and things look like? I think memories right now, for the large part, are mostly useless. I think the kinds of memories that you might want the coding agent to hold on to are specific quirks about how your team works in the project or certain rules like only use like camel case, for example, it's better to place those sorts of things in like a general sort of like guideline or rules file, for example. But I found that this idea of like asking the agent, at least coding agents to like hold on to certain memories about the project or like how you work or things like that, or you mostly have to like force it to store those things into memory.

1:08:15And I don't think people, they don't want to have to think about those sorts of things. So it's something we're thinking about is how can we hold on to the tribal knowledge that these agents learn along the way that people aren't documenting or putting into rules files without the user having to go out of their way to sort of force them to store these things into a memory database, for example. Those were like kind of like workspace rules or tribal knowledge, like general patterns that you use as a team. But then there is like in our, we ran this like internal experiment where we built this to do list tool, where there's only one tool where you could just write the to do.

1:08:57And every time you could like rewrite the to do from scratch. and we would passively as part of every, like not every message, but like every once in a while, we would pass in this context of what the latest state of this to-do list is. And we found that that actually keeps the agent on track after multiple rounds of context summarization and compaction. And it could all of a sudden build like an entire complex kind of task from scratch over 10x the context window length. And in internal testing, this was very, very promising. So we're trying to flesh that out. And I think something like that, we had earlier versions of the memory bank, which actually are like Nick, Nick Baumann, our marketing guy, came up with this memory bank concept where it was like this client rules where you would tell a client like, hey, whenever you're working, have the scratch pad of what you're working on.

1:09:53and this is like a more built-in way of doing that and I think that also might be very very very helpful for the agents to just have like a little scratch pad like hey what have I done so far what's left specific app file mentions like what kind of code we're working on general context and passing that off between sessions yeah any thought on CloudMD versus AgentsMD versus AgentMD I built an open source tool called Agents927 like the XKCD that just copy-pastes it across all the different file names so all of them have access to it? Do you think there should be a single file? There's also the IDE rules versus the agent rules.

1:10:34There's kind of a lot of issues. I actually think it's fine that each of these different tools have their own specific instructions because I find myself using a curse rules and a client rules separately. When I want client, the agent, I want him to work a certain way that's different than how I might want Cursor to interact with my code base. So I think each tool is specific to the kind of work that I do and I have different instructions for how I want these things to operate. So I think I've seen a lot of people complain about it and I get that it could make code bases look a little bit ugly, but for me, it's been incredibly helpful for them to be separated.

1:11:08I noticed that you said him. Does Klein have a gender? Yeah. Okay. Does he have a whole backstory? Yeah. Personality? So Klein is a play on CLI and editor. Because it used to be Claude Dev and now it's Claude. Yeah. I feel like Claude kind of stands out in the space for being a little bit more humanized than something like a Cursor agent or a co-pilot or a Cascade. And I think there's Devin, which is a real name. Claude is a real name, I guess. Claude is a real name. Yeah. Yes, I think we've all been intentional about just sort of humanizing it because at least in working with it kind of gives you more confidence in it and that I could lean on it a little bit more.

1:11:47There's kind of a trust building, I think, with an agent. And the humanizing aspect of it, I think, has been helpful to me personally. This goes back to the narrative integrity. It's actually really important, I think, to anthropomorphize agents in general. Because everything they do is like a little story. And without having a distinct kind of identity, you get worse results. And when you're developing these agents, that's kind of how we need to think about them. We need to think that we're crafting these stories. We're almost like Hollywood directors. We're putting all the right pieces in place for the story to unfold.

1:12:26And yeah, having an identity around that is really, really important. And Klein, he's a cool little guy. He's just a chill guy. He's a chill guy. He's helping us out. He's always happy to help. Or if you tell him to not be happy, he can be very grumpy. So that's great. Awesome. I know you're hiring. You have 20 people now. You are aiming to 100. You have a beautiful new office. What's your best pitch for working client? A lot of our hiring right now is, so far, it's been just friends of friends, people in our network, people that we've worked with before that we've trusted and that we know can show up for this incredibly hard thing that we're working on.

1:13:09And there's a lot of challenges ahead. And I think the problem space is probably the most exciting thing to be working on right now. Engineers in general love working on things that make their own lives easier. And so I couldn't imagine working on something more exciting than a coding agent. And, you know, it's a little biased, but I think a large part of it is it's an exciting problem space. We're looking for really motivated people that want to work on challenges, like figuring out kind of like what the next 10 years looks like and building kind of the foundation for, you know, what comes next after background agents or multi agents and really help in sort of defining.

1:13:45how all this shapes up. We have this really excited community of users and developers. I think being open source has also created a lot of goodwill with us where a lot of the feedback we get is incredibly constructive and helpful in shaping our roadmap and the product that we're building. And working with a community like that is one of the most fulfilling things ever. Right now, we're kind of in between offices, but doing things like go-karting and kayaking and things like that. So it's a lot of hard work, but we make sure to have fun along the way. Klein is a unique company because it really does feel like we're all just friends building something cool.

1:14:23And we work really, really hard. And the space is not just competitive. It's hyper-competitive. Capital is flowing into every single possible competitor. We have forks of forks, like I said, raising tens of millions of dollars. And we're growing very rapidly. we're at 20 people now we're aiming to be at 100 people by the end of the year and being open source it has its own challenges it's like people we do all this research we do all this benchmarking work to make sure our diff editing algorithm is robust the way we're working with these models to optimize for the lowest possible diff edit failures and then we open source that and then we post it on twitter and someone's like oh thanks so much for open sourcing that i'm gonna go and like raise a bunch of money with like our own product with it but But the way that I see it is like, this is, you know, let them copy.

1:15:13We're the leaders in the space. We're kind of showing the way for the entire industry. And being an engineer and building all this stuff is super exciting. So working with all these people is just amazing. Okay. Awesome. Thank you guys for coming on. Yeah. Thank you. This has been so much fun.

1:15:40Thank you.