LIVE: Ambient Agents and the New Agent Inbox ft. Harrison Chase of LangChain

0:00Hi and welcome to Training Data. We are mixing it up for this week's episodes and dropping a conversation that was filmed live at our annual AI conference in San Francisco, AI Assent, with Langchain Founder and CEO Harrison Chase. We hope you enjoy this conversation with Harrison about the future of Ambient Agents and an email inbox for agents unfolding. Stay tuned for a few more special AI Assent drops on our podcast feed later this week.

0:27Everything we're about to talk about is what's next. And the first person up here is Harrison Chase, or very own Harrison Chase who's been at every AI scent. He's been talking about agents longer than anyone I know. We'd love to have you come up here and talk about agents and particular ambient agents, this new concept that you guys at Lane Chain are bringing to the world.

0:51Thanks for that intro and excited to be chatting. My name is Harrison, co -founder CEO of Lengchain. We build developer tools to make it as easy as possible to build agents. A lot of the agents that we've seen being built so far are what I would call chat agents. So you interact with them through a chat interface, send them messages they run and respond and that's great. They're great for a lot of purposes. But one of the concepts that I'm really excited that I did about is the concept of ambient agents. So what is an ambient agent? The way that I like to define an ambient agent is ambient agents listen to an event stream and act on it accordingly, potentially acting on multiple events at a time.

1:27And so what are the differences between this and normal agents? So there's a few. One, what are the triggers? So it's no longer a human coming in and sending a message. It's an event that happens in the background. How many of these can be running? With chat, you can usually only interact with one agent at a time, maybe you open a few windows and you have a few running at the same time, but it's generally one with ambience -style agents because it's listening to these events. It's however many events are happening in the background, so it can be a far bigger type of number. Another interesting point is the latency requirements around it.

2:00So with chat, you message, you expect some response back pretty quickly or you get bored and you go to another website or something like that. Because these ambience agents run in the background, they're triggered by events. They can run for a lot longer period of time before you need a response in any shape or form. So there's generally much less strict latency requirements. And then lastly, I think it's interesting to think about the UX of these agents. So for these chat agents, mostly chat bots, that's a pretty familiar interface by now. I think there's a little bit of a question of how do you interact with these agents that are in the background because they are running without you knowing that they're running.

2:37But as they'll talk about it in a little bit, it's still really important for you to interact with them in some form. So just to make this concrete an example of an ambient agent could be an email agent that listens to emails coming in and acts on them accordingly and maybe tries to respond or maybe tries to schedule meetings or maybe pings you or pings other people on the team. So that's kind of like a concrete example of one type of ambient agent that we're seeing. So why ambient agents? I think they're interesting for a few reasons. First, they let scale ourselves. So if you interact with a chat agent, it's generally one -to -one.

3:08you're doing one thing at a time. When you have these Ambient agents, there can be thousands of them running in the background. And so that just lets us scale our impact a lot more. Two, they can get at more complex operations. So when you're interacting with that chat agent, it's generally because of the latent requirements, it's generally a simpler operation that it's doing. So you might have the humans send a message, it goes to the chatbot, the agent, it responds right away. Maybe it calls a tool, maybe two tools, the more tools it calls, But longer it takes to run, you can't do that with Ambient Agents because you don't have this as strict lanes requirement, you can call a ton of tools and do more and more complex operations.

3:46You can add in other steps as well. So you can add in explicit planning or reflection steps and generally build up the complexity of the agents that you're building. One thing that I really want to highlight is Ambient does not mean fully autonomous. So I still think it's really important that we are able to interact with these Ambient Agents. and there's a few different interaction patterns that we see people building towards. So one is approving or rejecting certain actions that these agents want to do. If you want to have an ambient agents that's potentially giving refunds to customers who are emailing in, definitely when it starts, you're going to want to have a human in there approving some of those things.

4:24Second is a more advanced option of this editing the actions that they do. So maybe they suggest something you don't want to approve or reject it, but you want to explicitly edit it and have it do that. that third, these agents can get stuck kind of like halfway down. And so there should be an inability for you to answer questions that they might have, just like you would answer questions of a coworker if they're working on a deep problem or something like that. And then fourth, because these agents take a lot of steps, it might be very useful for you to go back to the tenth out of a hundred steps or something like that, interact with it there, modify what it's doing, give it some feedback.

4:57back. And so this is what we call time travel. And facilitating this is a cool new interaction pattern we see. So there's a few reasons that having this human in the loop is important. First, it just gives better results. So if you think about deep research, which isn't exactly an ambient agent, but it is a long -running agent, there's a period of time up front where it asks you some clarifying questions to go back and forth, and that generally helps produce way better results than if it just went off whatever your initial kind of question in her statement was. And so having this human in the loop in the form of deep research, asking these clarifying questions in the form of ambient agents, there's different types of patterns.

5:35This just gets better results. It also helps build more trust. So if you're doing explicit actions, like giving or sending payments or approving things, having the human in the loop just builds more trust. And then third, and this is maybe the most subtle one, is I think it helps a lot with the memory of the agent. So when I'm talking about memory, I'm talking about learning from user interactions. If you don't have the user interacting with the agent, then there are no user interactions to learn from. And so having this human in the loop helps inform a lot of the memory things that you want to be building into the agent so that it can do better in the future.

6:11And so with this importance of the human in the loop, I think it's interesting to think about what a good UX for this might look like. This is one thing that we've kind of built as a prototype at Langchain, which is the concept we call an agent inbox. It's an inbox for your agent to send things to you. You can see when it requires actions. You can see some descriptions. If you click into a row, you can then see a more detailed description of what's going on, what explicitly it wants approval for, or whether you want to respond to it. And there's a few different interaction patterns here. Talking a little bit very briefly about some of the things that we're building that we think help was this.

6:47We've paid a lot of attention in Lengrap, which is our agent orchestration framework, to make it good at ambate agents. In particular, we've played a ton of attention to the persistence layer. That backset. This enables a lot of these human interaction patterns because basically you can run your Lengraft agent. You can stop at any point in time. The entire state as well as previous states are persisted. And so then you can have all the human and loop interaction patterns. You can wait for a second, a day, an hour, however long have the user come in, see the state, modify it, go back to previous states, things like that.

7:19We're spending a lot of time right now on Lengraft platform as infrastructure for running these agents. These agents are often way more long running. They're often bursty because they're triggered by events so you could get thousands of events at a time so you need to be able to scale up. And they're flaky in nature, not just because of typical software things, but also because of this human and the loop pattern. You want to be able to correct mistakes. And then finally, we're building Langsmith as well for these agents. They're really long running. They can often mess up. they're doing more complex things, having visibility and observability into what they're doing is really, really important.

7:52As a concrete example of this, one of the things that I built on the side is an email agent, so if you've emailed me in the past year or so, it's drafted a response or sent a calendar invite. It's still human loop. I use the agent inbox all the time. It's open source and on GitHub. So if you want to see how all these components come together and what I think is a pretty cool and unique and hopefully glimpse of what's next. I would encourage you to check it out. And with that, I will hand it off. Thank you very much. Thank you very much. Thank you. Thank you very much.