Harrison Chase of LangChain on Deep Agents, LangSmith, and Earning Trust | NVIDIA AI Podcast Ep. 297

0:00And so I think like these always on asynchronous event driven agents, that will be a really big productivity unlock. And especially in enterprises, there's so many events that are just triggering, triggering, triggering. And so if you can have agents listening to those and firing off, I think that will be a massive game.

0:17Welcome to the NVIDIA AI podcast. I'm Noah Kravitz. Our guest today is Harrison Chase. Harrison is CEO and co-founder of LangChain. One of just the most incredible stories of this whole generative AI era that we're in, as Harrison will get into in a minute, LangChain was founded about three years ago, over a billion downloads. The whole point is to help developers build applications with LLMs and now getting into agents and agentic frameworks and all that great stuff. So we're going to get into it in a moment. Harrison, thanks so much for joining the AI podcast and welcome. Thanks for having me.

0:52Excited to be here. So let's start about three years ago. you started Langchain with this premise of building tools so developers could build apps with LLMs. What did you see back then that either others didn't, or even if they did, you saw and just thought, this is where things are going, this is where I'm headed? What got us really interested was seeing the applications that people were building on top of LLMs and the systems they were building around the LLMs in order to power those applications. And those systems had a lot of similarity with each other, even early on. And even early on, we could tell that they would get quite complex over time.

1:32And so a lot of what we built is tools to help people build these systems, these agents, which we now call agents, around these LLMs, and figuring out what the common patterns are and the common tooling is and making it really easy for anyone to do so. Right. And so now you've coined this, I don't want to say this term, but Lang Chen, you talk about deep agents. Yeah. Yeah. So what is a deep agent? And then maybe we can get into talking about the enterprise and why would an enterprise in particular care about that distinction? Yeah. So about a year ago, we saw a few really interesting things. So maybe even backing up like, you know, three years ago.

2:07Great. Like LLMs, you want to connect data. You want to connect these other things to them. Fantastic. How do you do that? Turns out it's really hard. And the best way to do that for for different types of agents was actually pretty different. You would build different scaffolding. You would build different workflows around the LLMs. about a year ago, we saw Cloud Code come out. We saw Manus come out. We saw Deep Research come out. And under the hood, all of these had the same kind of general architecture. They were simple in some ways. They were now running in a loop calling tools, but then they also had common patterns of connecting to a file system and having sub-agents and doing planning.

2:41And so about nine months ago, we released for the first time Deep Agents, which is a library, and we've been building it ever since. And we've just continued to see kind of like this same pattern of giving the LLM more autonomy in this environment for interacting. This is what powers OpenClaw, for example, is this type of harness. And so Deep Agents is really this new type of agent harness that we think is really general purpose and that you can customize to do different things. But it's not like you're reinventing the scaffolding each time. You're just customizing it with prompts or tools. And so it's way easier to get started with and also way more powerful because it's a simple thing under the hood and simple is really good.

3:21And so Deep Agents is this general purpose, agent harness, model agnostic, open source that we've been building for a while. And we're starting to see more and more agents build on top of. So when you're working with customers and the enterprise in particular, and we're getting into these systems that are so powerful, becoming so powerful, in large part because they are autonomous to a larger degree. And as you said, they can do more now. The agents can control the screen and go off and do things with apps and such. What are your conversations like with enterprise leaders? And what's kind of the feeling around, is it a tension between risk reward?

4:01Is it just the excitement for what the systems can do? And so there's trust in building these systems that give agents more leeway. What are those conversations like? There's a lot of things. So one, like, you know, not everything needs an autonomous agent. Sure. And so one framework we have, LaneGraph, is really good for when you actually want to combine some of the autonomy of LLMs with more directed workflows and more control. And so honestly, when talking with a lot of enterprises about deep agents, some of them are just like, we love LaneGraph. LaneGraph is better. We're going to stick with LaneGraph.

4:31Sure. And that's fine with us. We think there are different use cases and different things. But that's definitely kind of like one component that comes into it. another component that comes into it is definitely just like okay great like the LLM is doing a bunch but how do we know what's going on sure and so another thing that we work on is Lang Smith which is observability and evals and that's basically our answer for that you there's this really interesting thing about agents compared to software where where agents the the interaction space for agents is way more open-ended you can ask it anything like text is infinite if you have a UI there's a bunch of different buttons you can click and so it's much more constrained.

5:05And then also models are not robust at all. Like, you know, they're non-deterministic and then you change one word and the answer changes completely. So this is why we think observability is really important. And that's a huge thing that enterprises care about and very related to observabilities than evals. Because sure, you can see one thing that happens, you can tell why it goes wrong, but what if, you know, what if you want to test how it did on like 10 different questions, 100 different questions. And so building up these eval data sets is a big thing we work with folks on. Right, right. And so Langsmith is the platform for building agents as well as observing and evaluating?

5:36Yeah. So the way that we think about the agent development lifecycle is build, test, run, manage. Okay. And so the build is all the open source. It's like choose your fighter, choose LangGraph, choose deep agents, choose another framework. All of our stuff works kind of modulary. But then this test, run, manage, that's Langsmith. So we've got a bunch of stuff around testing and evaluating these models. We have a deployment platform for deploying these at scale. And then we have observability and other things for managing them. Let's talk about skill, or maybe you can talk about skills for a minute.

6:07When I first started playing with these tools, and I'm not a developer, I'm just kind of a technical layperson, if you will. I love playing with these things. Back in the day when they first came out, maybe it was baby AGI, whatever it was, I spun my computer right into the ground in an infinite loop. But when I first discovered skills, it took me like there was a moment where it sort of took me back where I was like, wait, I just describe it and it goes off and it builds. And then, of course, it does, because that's how this all works. But can you talk a little bit about skills and about kind of that that same idea of giving the agent the autonomy to write the tool and run with it?

6:46But how do you keep it in check and keep things secure? Yeah, skills are a great way to package up knowledge and other kind of like instruction sets and other tools for an agent to use. And so they started in coding agents and a skill would involve basically a markdown file with some instructions and then some scripts that you could run. And one of the things that's kind of become clear over the past few months is like coding agents are very general purpose in a lot of ways. And so this same idea of a skill as a markdown file and then some scripts to run is really, really interesting. We see a bunch of different types of skills.

7:19Some of the skills are purely kind of like informational. So like you want to learn about something great go go read this markdown file. Other skills do things and this is where it starts to get I think like really interesting. It could be a Python script that that hits a URL. It could be a Python script that runs some GPU accelerated compute. And so this is this also ties into the environment aspect. So when we think about agents we think of a model a harness and this is deep agents and then an environment that it runs in a runtime for it. Right. And so So NVIDIA just released OpenShell, which is a secure runtime for it.

7:51And then the other thing that's related to the runtime is also like where it runs. Does it run on a Mac mini? Does it run on some GPU accelerated environment? Does it run in the cloud? And so those three components and being able to pick and choose what you need for different jobs is a big part of kind of like customizing your agents. Was there a moment, or can I put you on the spot and ask you to think of kind of an aha moment where this, the idea of deep agents really clicked and in a use case and whether it was something internal at Langchain you're working on or maybe with a customer, was there kind of an aha moment where you were like, yeah, this is it?

8:26I think so it started just by seeing really the three things of Manus, deep research and cloud code. And this is the same way that LangChain started as well. Just going to early meetups, seeing things that people were building and seeing patterns. Right. And so the first version of deep agents, just like the first version of LangChain, I hacked on over a weekend. And it was a weekend project. I'd been talking internally with some folks and being like, oh, like, you know, cloud code is really interesting. Like Manus, they've got some similarities. And so it wasn't until I had time to kind of like sit down on a weekend and hack some stuff together that we came up with a few patterns of what these similarities actually were.

9:02And then using it, I think the first thing we used it for was a deep research type thing. And so we gave it access to a bunch of files and we just put it in this like virtual file system and had to do some research. And it wasn't even really doing rag. It was just grepping and globbing like a coding agent would over these files. And it worked fantastically well. And so I'd say deep research was the first concrete thing, but really the idea came from just seeing a pattern and spending a weekend kind of like hacking on it. You mentioned earlier the importance of, I don't know what the words you use, but auditability, traceability, being able to see how do the agents do what they did.

9:39Can you talk a little bit about evaluation-driven development and how that plays into, and again, in the enterprise, building that trust in what the agents are doing? Yeah, if you talk about trust in an enterprise, what does that mean? That means that the agent's doing what you want it to do. There's a few different ways that we see people getting that trust. Part of it is observability and traceability and being able to go into an agent run and see exactly what steps it took and exactly what it did. The other part where trust comes in is having these scenarios and running the agent over them and seeing how it performs and evaluating that.

10:15And this is kind of like what we talk about as evaluation-driven development. You come up with these scenarios ahead of time. One common misconception here, by the way, is that you need like 1 ,000 scenarios for it to be effective. You could start with five. You could start with 10. It really doesn't matter. I think creating these evals is a really good way to do product thinking about what the agent should act. because this is another thing like agents can do anything yeah but they shouldn't do everything they should do like what you want them to do and so being able to be like being forced to come up with like hey these are 10 questions that we expect the agent to get asked this is what we think a good response is for each of them this is what a bad response is for each of them that's a really good kind of like mental model for kind of like coming up with what these agents should do and then you can use that to drive all of your changes so you change a prompt great you can run it against this benchmark did it improve did it did it did it get worse and then this this eval data set is living over time as well.

11:07So as you release it to first like a small set of users, you might see them using it in unexpected ways. And then some of those ways you might be like, okay, maybe they shouldn't be doing that. Let's put some guardrails around it. But other ways, you might be like, yeah, that's totally legitimate. We had no idea they would use it. Let's add some data points to our eval data set. So when we go and change the prompt in the future, we can make sure that it's still good at these use cases. Are enterprise customers open to kind of rolling with that, you know, oh, we weren't expecting this behavior necessarily, but it's good behavior.

11:36And so, you know, is there a sense of kind of experimentation? Obviously, in the AI community and the open source community, it's all about experimenting and sharing and things are going so fast. Is the enterprise embracing that at all? The best ones do in limited ways and with a limited blast radius. They might roll it out internally, for example. They might roll it out to a set of like alpha customers. They might roll it out to 1 % of users or something like that. there's definitely way more caution there than there is with gen ai native startups but but building agents is so iterative and the importance of this iteration can can really not be understated and so i think the the enterprises that are i think a failure mode for enterprises is you have some idea of an agent you take three months to craft a bunch of examples you take another three months to to build the agent you take another three months to get humans to look at everything but the space has just moved so fast like the the whole the whole idea you came up with there's probably a better like there's just a better way to do it at that point and so i think like you have to kind of ship you have to learn you have this is another thing by the way that no one likes the answer for but like you have to you have to basically redo your agent every every nine months at the pace that things have been like with these agent harnesses if you're using an agent architecture from like a year and a half ago you should very strongly be considering looking at rewriting on top of an agent harness or something like that.

13:02Right. And again, we're for performance only or for, yeah. Yeah. For performance that that's still the bit like, um, that, so, so there's two things. It's like performance, but also scope of what the agent can do. So if the agent's doing a very small thing, it's not as valuable as if it's doing a big thing. And maybe like a year and a half ago, you just couldn't get it to do the big thing. So you focused on the small thing, but now you can. And so if you're not like reevaluating that and saying, Hey, there's this big thing, let's hook up an agent harness. Let's take a stab at that. You absolutely need to be doing that.

13:29So I want to ask you about models. Frontier models, I would say everybody, but I think the kind of mainstream AI world focused on the latest and greatest and what can they do and everything. Open models have become incredibly important. I mean, they've always been important, but I feel like the past year or so, incredibly important. You spoke earlier about OpenClaw and NVIDIA's OpenShell and the Nemetron family of models. How do you approach and how does Langsheng approach and then your customers mixing Frontier and OpenModels together to achieve cost-performance ratio and all manner of other things?

14:12What's your approach on mixing those? Yeah, I think there's a bunch of different ways that we combine them. So I think one obvious way that we worked with NVIDIA on a Blueprint for is with deep research, you have a bunch of subagents. And those subagents might want to be specialized agents. And there might be an orchestrator kind of like agent that's using a frontier model. But then when it goes to a subagent, it might want to use either like a fine-tuned model or an open source model for cost or latency reasons. And so when you have these big adjentic systems with these subagents, it's totally possible that one part could be using a frontier model and one part could be using an open source model and another part could be using a fine-tuned model um the other we've been paying a lot more attention to open source in the past even just like two weeks i would say for probably two reasons one um i think they're getting good enough to where they can drive this harness so so you know the like being able to properly utilize everything in the harness is not it's not super easy and for a while it was only the frontier models that could do that we're starting to see they're still a step below the frontier but we're starting to see that these open source models can drive the harness which is really interesting because this is the most agentic stuff.

15:17And then the other thing that's causing us to look really hard at open models. If I could stop you for a second to hear us and back up, what are the qualities that a model needs to drive the harnesses successfully? So at the risk of signing a little broad, it needs to be intelligent. It needs to be good. Another thing that is maybe underappreciated is it probably needs to be good at coding. So we've actually seen that QuenCoder is a better general purpose model than just the Quen series of models. because a lot of what makes up this harness looks very similar to coding agents. So this harness has a file system.

15:50It has a bash tool, right? So if the model knows how to use it, if it's a coding model, then that's actually really, really good. And so I think models that are better at coding are generally actually better general purpose agents. Yeah, no, that makes sense. And so then the sub-agent models you were talking about. Yeah, and so then a second thing that made us look at this, look at open source models even more is OpenClaw. So there's a bunch of really interesting things about OpenClaw, but one of the interesting things is it's always on. It's proactive. It's running. And so if, you know, if you're using a coding agent and you kick it off, even let's say like 20 times a day, you know, you're probably OK paying like some good amount for that.

16:24If it's running every 10 minutes, like, oh, oh, my God, you cannot. And if you if you're running like three of these, you just cannot do that. And so I think like cost is a really interesting reason for these open models, especially in these proactive, always on scenarios to make them become popular. Um, shifting gears for a second, uh, Langchain just opened NVIDIA for the Nematron coalition and Langchain joined. Can you talk a little bit about, um, why and what it may or may not mean going forward for Langchain users? Yeah, we, we need open models and we need harnesses that they can run in. Um, and you know, we, we think we can provide the harness and we want to work with NVIDIA and all the other companies in the coalition to, to help provide a model that can that can work with that harness and others as well um i think like you know as we talked about like the open source models are getting good they they're still a little bit behind the frontier models in terms of driving the harness and so great we can use them in subagents you know we can maybe use them um uh for some of these kind of like triggers and the always on but if they can drive the really expensive workloads i think that's going to be really transformational in terms of um what you can do with open models which generally mean what you can do with more sensitive data, what you can do more cheaply, what you can offer to customers just more.

17:44And so, yeah, I think at a really high level, we're excited about the NVIDIA NemoTron Coalition because we want an open model that works really well with open harnesses. And then a third part, which actually I don't think was part of the coalition when it started, but I think the open runtime is really important as well. And you guys are also doing stuff around that. This is my favorite question to ask, and I'm sure the hardest, but maybe the most fun to answer. What's next? What do you think agents, agentic systems, Langsmith, Langchain, the company for that matter, is going to look like in, and I'll let you kind of go with what timeframe makes the most sense because I ask and depending on the guests, they're like a year.

18:25No, that's too long. No, no, no, no. But what do you think is coming down the pike as far as, you know, agentic systems and all of these things that you're working on every day? I'd maybe call out kind of like three things that I think are interesting. One's pretty short term, and I think we'll see in the next like month or two, if not by the time this comes out, but asynchronous subagents. So right now, when an agent kicks off a subagent, It basically waits for it to respond. And that's great. But if these subagents start to get really long running, you want to just have them run in the background.

18:58And you want to have this manager orchestrator agent like check in on them and maybe update them. And so I think one trend that we'll see is encoding right now in coding agents, you interact with the agents that's doing coding. I think we'll start to see a trend where you interact with this orchestrator agent and that orchestrator agent spins up a bunch of background coding agents. And you just talk to the orchestrator and say, hey, what's going on with this experiment? What's going on with this feature? And so I think we'll start to see asynchronous sub-agents become a bigger and bigger topic.

19:25I hate to resort productivity. But how much of a, is that going to be a step change? Or how much of a difference in terms of what you're able to accomplish? So I think this bill, like the only reason asynchronous sub-agents even make sense is if the sub-agents themselves actually run for a while, right? Like if they just run for like one second and then return, you can just make them synchronous. and and and so i think like it will be a productivity gain but i it like requires these agents to be long-running in the first place and and i think that's the real productivity gain and i think this is just a nice interface on top of them what one thing um that i that wasn't on my list of three things but i think will also be more and more impactful is basically these agents being proactive running in the background always on listening to events that i think will be a massive productivity gain so i have an email agent it runs in the background it listens to my emails um when when it wants to respond they're still human in the loop but it like it like flags a draft it's like hey here's a draft do you want to approve it do you want to change something that is so much more efficient than if i had to go like there's no way i would take an email copy paste it go to chat gpt say hey can you draft me a response copy paste that like that and so i think like these always on asynchronous event driven agents that will be a really big productivity unlock and especially in enterprises there's so many events that are just triggering triggering, triggering.

20:43And so if you can have agents listening to those and firing off, I think that will be a massive game. The other two things that I think are coming down, one, agent memory. We started to see this a little bit with open call, but I think the idea that it could remember things as you interact with it, it could actually update its own tools and skills and description itself. I think more and more we'll see agents kind of like remembering things and learning from their interactions. And that's why human in the loop is important as well. That's why I don't think these things will be fully autonomous because they need to learn.

21:13And the only way you do that is by interacting with the environment, with humans. And so I think that'll be a big piece of it. And then the last thing is agent identity. So, you know, if there's an agent in an enterprise and I chat with it and you chat with it, whose credentials does it use? Does it use mine? Does it use yours? Does it use a fixed set? So previous to OpenClaw, I think we saw that basically everyone was doing the on behalf of model. So the agent would act on behalf of me, on behalf of you, on behalf of the end user. And I would pass like my Slack credentials through. And so I might get a different answer than you would get.

21:45I think the thing that OpenClaw changed is people started thinking of these agents as like identities, as their own things. And I think we'll actually see more things where they will be like, hey, Tom is a marketing agent and you can chat with Tom and I can chat with Tom and Tom has a persistent memory and Tom has its own credentials and Tom can go and do things. And Tom is Tom. Tom is not acting on behalf of me or you. Tom has its own accounts with Slack or Gmail. That's a big thing that we need to figure out that I don't think anyone in the industry really knows. I was chatting with one SaaS provider.

22:16In all the open-claw craziness, they were making it really easy for people to create accounts for their agents. But it's still like an account. And so will we just see more and more people create normal accounts? Will there be special agent accounts? I don't know. But I think this idea of agent identity is really interesting. Yeah. There's a whole can of worms on the other side of the words agent identity, I think. But not for this conversation. So, you know, you mentioned the weekend project that you worked on that unlocked things at Langchain for you. OpenClaw, another weekend project, went incredibly viral incredibly quickly.

22:51What are your thoughts or how has that impacted the work you do? And I'm thinking more about the perception that users, developers, enterprise customers might have about engines. Was there a rush of people knocking at your door saying, hey, can you build me a claw? How has it changed things? A hundred percent. I mean, I think Jensen said, what do you say? Every enterprise needs a claw strategy or something like that. And we're absolutely seeing that. I think it's set a North Star. It's set a new objective for what these agents can and should be able to do. Now, there are a lot of things that you probably want to do more securely than in OpenClaw.

23:30the whole reason it took off is because it can, it can do everything. And that's great for, for weekend projects and hobbyists. But when you bring it into an enterprise, you're understandably going to want more, want more control. That's why we're thinking about agent identity. That's why we're thinking about observability, but in terms of like, did it change the North star for, for what we build? Absolutely. It did. I think it also made it really, it made it so much easier to communicate some of the ideas as well. And so that's, that's been fantastic as well. Amazing. Harrison, so much we just talked about in a short amount of time and so much more, but I'm sure by the time we cross paths again, as you mentioned, you take three months to scope and three months to build and all of a sudden it's nine months and no more.

24:11So the next time we cross paths, I'm sure it'll be a different looking world, but kind of built on these same things. But for folks who've been listening or watching and want to learn more about Langchain, the work you're doing, best places to go online, website, socials, research blog, anything like that? Yeah, we have a great blog. It's blog.langchain.com. A lot of the stuff we talked about around context engineering and agent identity will be blogs on there. And we update that a lot. And then Twitter. I think everything in AI is happening on Twitter. We're just laying chain on Twitter. And so you can find us there.

24:44Easy enough. Harrison Chase, thank you so much. It's been an absolute pleasure. Appreciate you taking the time to join the podcast. Thank you for having me.