AURA and Open-Source Agents for Production Operations

23:38Typically, we see like, you know, 70 % confidence is what we see a lot of folks kind of define initially there. But, you know, I'm not having to think about like how the tools are provided to those sub workers. I'm not having to think about how that planning and self correction loop is defined for the orchestrator. I'm not having to think about like how the orchestrator provides the relevant context to just the workers that it's currently interacting with. What is the communication path back forward and back? Is it just dumping files in place for them? Is it standard in and standard out? Can it hand off more complex things?

24:13If I have a sub agent that pulls together a database for me, can I hand that to a different sub agent? What are the tools available in the tool chest? Yeah. So we kind of go for a single path in, single path out for right now. That'll change as the harness develops. But, you know, it's meant to be, hey, the orchestrator does the planning and then hands it off to the workers and the workers hand it back. And then the orchestrator can hand it to different workers if it determines that, hey, I've built some context, but I need to clarify it. I need to get a little bit more context, execute another tool, or maybe execute the same tool again, because it didn't give me the result I was looking for.

24:46And so the orchestrator becomes kind of the important part of that process. But the thing that we're starting to see more and more of is like, And, you know, these multi-agent workflows, you know, they have a lot of tendency to become really complex really quickly. And so we wanted to make sure that was simple on day one. But like, as we see more and more of these patterns emerge, we definitely want to make sure that Aura kind of enables you to like gain the benefits of either like performance or reliability out of what that like orchestration flow looks like. One thing I'll also call out with orchestration, which is pretty interesting, is that, you know, when you think about like how you are gaining better economics out of the tokens that you're using in some of these workflows, again, you're dealing with scale.

25:26And so like, you want maybe the bigger model, maybe you want an opus on how you're doing the planning and the reasoning and the top level orchestrator. But then on those sub workers, you can actually declare a different LM a different model behind the scenes that, you know, operates as more of a delegated task worker, right? Like, hey, maybe I want to use Haiku, maybe I want to use an open source model. And so that actually is also something that Aura makes insanely easy to do using our declarative format so that you can say, hey, I want to not only use a multi-agent workflow, but I also want to use smaller agents for the workers and ultimately get better kind of like token economics out of the whole process.

26:02And that's kind of like one of the things that we solved on day one that we've seen some pretty surprising, like pretty exciting results out of just enabling that to happen and kind of embracing the idea of a multi-agent system with a lot of different models in the mix. Totally. Yeah, there's a lot of like classic like data categorization, extraction, etc. that is trivial to do with a small model. And you really don't need to throw opus at it, but you want that as a sub piece of your broader thing. So yeah, making that easy is super helpful. So to playback, making sure I understand, you're sort of right now operating on like sort of single orchestrator set of available workers, the orchestrators deciding what and just sort of spawning things out.

26:42And then it can dynamically decide, hey, I got this from here, but actually I want to send it over there. I'm doing things, but it's kind of single level of delegation. Is that exactly? Yeah, exactly. And we're to your earlier point, too, where we're making it as simple as possible. A lot of our interactions are just utilizing flat files on the system. And so, you know, that's something that pretty much every LLM, every agent is good at interacting with is a file on disk. And so, you know, we're not introducing a lot of like, you know, custom processes. We're kind of operating on the simple, most reliable thing that when it breaks, you know why it breaks.

27:15But ideally, it just doesn't break because it's a simple enough mechanism. I want to come back to talking about memory and human in the loop because those are interesting. But that points in another direction that I think is really interesting to explore, which is like the observability of the agents themselves. Like we've been talking about, you know, how you're using these to tackle observability data and things like that. But like what tools does Aura have for making it self-observable so that, as you said, you know, when it breaks, you can tell why it broke and how to fix it. Yeah. Being an observability company, this is like one of the things that we made a very hard requirement for us to solve.

27:48Basically, you know, even when we didn't GA Aura, when we were kind of in a pre-release, like an early access type of thing, we instrumented everything in Aura natively with open telemetry. So we're heavily invested in like open telemetry for the agents. We define everything again in the readme. You can see exactly how we're instrumenting that, what pattern we're using for open telemetry, which were built on top of open inference for open telemetry. And so you can see exactly in the spans of that trace of the agent interaction, you know, hey, how is it doing the reasoning for the next step? Was the tool call successful, etc.

28:21So we organize that automatically and we build on top of OpenTelemetry to make that happen. And the other thing that we have natively built into Aura is a really rich set of server-side events that can be returned back to the client that can really show you what's happening live as part of the investigation process or what is the agent doing behind the scenes. And so, you know, definitely want to didn't want to get to the point where it's like, hey, I give this really complex task to maybe a multi-agent workflow and I just kind of like wait on the spinner or anything like that. Like as that's executing, we have a CLI built into Aura so you can very easily just like start interacting with the CLI.

28:57And that is, you know, it'll very easily show you all those different events that I just mentioned. So you can do slash stream, you can do slash expand, and it'll show you like, hey, what are all the activities that I can see from the agent happening behind the scenes as I'm interacting with it? And then because it's built on top of open telemetry, I can send that to my observability system. And that makes it really easy for me to understand the auditability of the interactions from the agent, what did it actually do and why. So the reasoning steps across each process are also captured in that trace.

29:29And so we just wanted to make sure that, you know, one, it's built on open standards. You can send it to anything that supports open telemetry. And it becomes a great way for you to start building trust in your agents, right? Again, like this is a big barrier in the industry. I'm saying is that, you know, for SREs, like anybody who's dealing with like a production use case, you really have to understand exactly what's happening behind that agent in order to trust it. Otherwise, you know, you're not going to put it into production. You're not going to, you know, offload more tasks to it. you're not going to really integrate it into your workflow.

30:00And so the observability or the transparency of what's happening behind the scenes, like that was a insanely important piece that we wanted to solve on day one. One of the big patterns I'm seeing right now, once again, doing the metaphor to coding agents, because that's where I live a lot of my time, is like looking at traces, essentially, of how are things going and using that to iteratively improve. And if you're talking about an observability agent or an SRE agent, it needs the same thing, but you can't just be looking at files on your laptop anymore. you've got to have those deep observability traces.

30:30Totally. Yeah, exactly. Because we're built as an observability tool, our platform enables you to send those traces directly to us. So obviously we have the ability to pull those in, visualize those, show you the different service mappings and interactions. Most people have an observability tool in place. And so it's easy for you to take Aura and just point to whichever open telemetry provider you decide to use. The visibility that you're going to get into those workflows is not going to be limited to you using our commercial product. Let's come back and talk a little bit about memory because I think this is also a really interesting domain.

31:05And I'd love to get, once again, one or two levels deeper on how you approach it. As you commented, a lot of memory implementations out there are pretty naive. I'm going to use an even simpler example. You look at ChatGPT memory or something. And it remembers everything. It doesn't give me any visibility into what it remembers or why. And, you know, when I'm working on some thing for my son's project, it says, hey, because you're a director or you're a VP of engineering, you should be thinking about it this way. And it's just this one big undifferentiated mess. Obviously, that's not what you want in a production SRE context.

31:43You mentioned examples of exposing incident logs or incident histories and things like that. So like, how does your memory architecture work? Once again, first from a user perspective, and then like diving in under the hood? It's a good question. And, you know, a lot of this you can actually see in our PR is like how we're developing the next iteration and what we're already starting to improve within Aura. But the way that we look at it is like there's different levels of memory. And so, you know, there's memory like more turn-based or session-based basically, like that you want the LM to be able to recall, you know, what was the last tool call that happened on the last couple seconds, right?

32:18Or what was the previous run of this agent able to come to as a conclusion that maybe is right in front of the agent at that moment, right? And that's more of the turn-based or session-based memory, which is kind of like the lower levels of where we want that really fast. Kind of like when you think about memory architectures with like, you know, having DDR is like, you know, my RAM versus a hard drive versus, you know, something that's like off-site storage. We kind of took the same approach with Aura where there's very simple memory that's built for turn-based and session-based that, again, you can tune along the way.

32:49And then what we're looking at, like the really interesting problem is when we start to get into some of those like longer horizon memories. And this is actually where like, you know, we are working on this within the platform itself, within the Mesmo platform to offer this as something that, you know, is a little heavy to do in line, but it's actually great to do kind of as an async or background process that just gets exposed to the agent that it doesn't have to waste tokens on. And so this is where we're looking at, like, again, more of the kind of needle in the haystack incident over incident investigation, but like tailored to the types of workflows that S3s would deal with, where those memories are not just about the last tool call, or, you know, the last iteration of this agent running, but it's like, hey, what happened in the last six months?

33:32What happened in the last 12 months? What happened like with this service that I started to see maybe a little bit of service degradation and also a change that somebody pushed from GitHub and also, you know, started to see some funky stuff in the logs like that all becomes a collective memory that we're kind of creating as a simple briefing to provide to the agent that just makes it easier for that agent to either solve the next thing, you know, the next time better, or potentially just short circuit the process entirely and say like, hey, you know, normally, I would kind of come to this conclusion of doing this next thing, but I can see the last time I did this, this thing, and I can kind of compare that to ground truth in the background.

34:07So I can kind of validate whether or not that had the intended impact that I needed. And maybe that'll change what I do the next time that this happens, because now I've actually evaluated that in the background. And so that's kind of like where the longer horizon memory is like something that we're very interested in. Because again, if you look at like, you know, just taking either the graph, you know, memory MCP server that most people are familiar with that you can just grab off of a kind of the standard MCP servers or something like memzero or mempalace or any of the kind of very general mcp you know memory servers we wanted to make sure that that was again shortening the circuit to the outcome or shortening the path to the outcome where we know the types of tasks that sres need to solve for and we know how memory becomes really important to some of those longer horizon tasks and so we're just going to provide that on day one what does that end up looking like technically then is that a kind of rag-like implementation?

35:00Is it exposed via another NMCP or it's a tool that a separate tool that you can query? Or like, how does that end up actually playing out in the agent? Yeah. So the way that we're exposing it today is just taking like the simplest, again, lowest common denominator across the board, which typically ends up being another tool. And so, you know, the way that we look at it, like we call them investigations. And so as I'm handling either a new investigation or looking of previous investigations, that's just another tool available to the agent that we exposed through our MCP server. How that practically looks, right, is we're providing kind of a briefing as part of that tool call.

35:36So it's kind of like the summary and then like where the agent can go for the next steps. Something that, again, like because we're kind of focused on building on top of these, you know, lowest common denominators in the space, which a lot of this looks like, you know, just an MCP tool. You could plug what we're doing in the background for memory or what we're doing in the background for context engineering, like longer horizon things into Aura. And that's, you know, what we see is a great path to consume that. But you can also plug it into, you've already invested in that agent framework that's, you know, you've already gone through the glass to get to that outcome that we were just talking about before, or you want to plug it into Cloud, you want to plug it into wherever, you know, you can easily do that.

36:13We did experiment with a little bit. And again, you could see this in our PRs, doing things like providing a virtual file system to the agent, where we can kind of like seed in those memories without the agent actually having to do the tool call. And some of that's in kind of an experimental phase where we're looking at like, how can we better expose these memories that don't actually require the agent to do a tool call and is rather just like transparent to the agent where it's not wasting tokens. But some of that stuff is like kind of in a research preview. And again, you can see that in our PRs and kind of follow along with that.

36:42Yeah, no, it's super interesting. And maybe this connects a little bit to the context engineering question, right? of like, what sets of things do you want to embed in your initial context? So it just kind of knows it versus make it on demand, go and call it versus, yeah, have it in an explorable database or find, you know, some other way of finding it. And there's also one last thing that I want to make sure I also layer on here that's related to memory. We don't always call it memory, but it's like on the same exact path, which is things like institutional knowledge, right? So like more of the institutional memory or institutional knowledge, right?

37:19Like where, for example, every SRE team, right, has a set of runbooks that they have internally. Maybe that lives in Confluence, maybe that lives in GitHub, maybe that lives in, you know, insert your favorite place to store runbooks, right? And so that becomes incredibly useful grounding for the agent. And so, you know, we also expose a really easy way for you to either connect us to where those runbooks are currently stored. If they expose something like a vector database, it's really easy to plug that in and be use that as grounding for the agent. So the agent will always kind of ground and respect itself based on or respected the workflow based on what you've defined as the source of truth being your runbooks.

37:55We've also actually published a blog post on like, hey, if the agent has the ability to not only consume those runbooks, but as it starts to see things during its investigation process, how can it actually contribute back? And we've published a blog post on this, how we're actually doing this internally, where we have our runbooks stored in GitHub. We actually use Aura for a lot of our own SRE workflows internally. And as we identify things that like, hey, this workflow has changed based on what's in the runbook, instead of that becoming a human process where somebody is maintaining the runbooks, and it's only a one directional, you know, like consumption by the agent, we've actually seen a ton of really positive success with providing the agent a feedback mechanism to basically iterate on those runbooks, open a PR and say, hey, I did this investigation, and I need to update the institutional knowledge.

38:43And that's just like a really good example of kind of the opinionated memory where like everybody has institutional knowledge in some way or form. Yeah, well, I love that because it closes the feedback loop and it gets us to the last sort of pillar that you talked about that maybe we want to dive into, which is like the amount of human in the loop you have. And sort of you described here a version of human in the loop, which is like I did a whole bunch of stuff. But here, let me suggest to a human an improvement. But yeah, how do you think about, you mentioned, which I love this idea of having the sort of gradient from almost entirely human with copilot going on to completely automated.

39:23But like, what does that end up looking like? How do you end up configuring it in Aura? And where do you see different types of SRE tasks typically falling out in terms of that level of autonomy versus human run? It's interesting because, you know, the way that people are looking at human in the loop today is like very dependent on the types of systems behind what the agent is interacting with. Right. So that's like one parameter, one dynamic. There's also like how confident are you in the workflow itself? And then there's also like how confident are you in the data that has been returned through the workflow?

39:57Right. So there's an aspect of how you provide the ability for somebody to say, hey, these types of tools, which maybe are read-only or, sorry, write, they have the ability to write or they have the ability to do something that is kind of a one-way door. I want to make sure that I provide a very hard kind of barrier for the agent to be able to know that that is something that's insanely sensitive and it needs to ensure that it asks for help before continuing on. Like that's kind of one piece that, you know, we see everybody with some level of like, hey, there's remediation. But like, I need to make sure that there's always a human that hits the go button before it does remediation.

40:36Because I know that that's a, you know, one way door that's really hard to come back from. That's kind of like one aspect of like limiting or providing human in the loop on the tools themselves. Then there's like, you know, how far do you want the agent to go in the process before, you know, ask for a person to kind of let it continue on with go, right? So like, maybe you're not limiting access to the tools themselves. But you're limiting access to like how far in the process the agent can get to right like you can basically act as an assistant. But at the end of the day, the person who is implementing the remediation to what you've investigated is always going to be a person, I'm not going to provide you those types of tools.

41:13And it's always going to be, you know, you're, you're read only, you're only getting me up to the point where you're creating a nice package for an SRE to go execute a script, remediate a problem, but you're coming to exactly what needs to happen before, you know, that point become like a simple place for the agent to either determine that it needs to continue on or, you know, just completely hand off the process. And then the last one is, you know, where we start to really get into, I think the, like the true benefits that people are looking for, which is when I'm thinking about like a truly autonomous workflow, right?

41:45Where, hey, instead of S3s handling this intent, it's actually an agent handling this intent. That's really where we start to get into, you know, like an aspect of, hey, how do you use some of the data that we were talking about before with the observability data, right? And like grade that over time, where even if you think that that's a pretty good workflow, and you have the right data, and you have the right workflow to find, you want to make sure that it is tested a certain amount of times before you give that the full like thumbs up, continue on, operate autonomously, right? That's kind of like the graduation process.

42:16So like there's those like three different areas there. Now, how that looks, you know, specifically with an Aura, it's all driven through the config file, where it looks a little bit unique is like, as soon as we start to get into those like longer horizon things, again, where it's like, hey, I'm not just limiting access to the tool itself, but I'm telling the agent where it needs to go ask for help. That's where we're exposing a tool within our MCP server to basically provide to the agent. So like the agent doesn't need to go figure out, hey, you've told me that I need to go ask for help here, but like, I gotta go figure out who's on call.

42:47I gotta go figure out how to get access to Slack. I gotta go figure out how to do, you know, X, Y, Z. Instead of what we're doing is like, that's a process that we're seeing across, you know, a lot of S3 workflows that needs to happen. So like, we're just exposing a simple tool for the agent to be able to ask for help. And then what we're doing behind the scenes is like some of that correlation that I mentioned. Hey, who's on call? Who do I need to go ask? Trigger a notification via Slack. somebody give me the thumbs up and then provide that to the agent versus the agent again having to do that itself so that's kind of like how we're not only defining the different types of human in the loop in the process like governed autonomy is kind of like what i what i think about it but also make it so that our platform kind of provides the simple mechanisms for the agent to kind of like ask for help from a single place without having to bloat a bunch of context to go figure out all the possible places that it could go ask for help and so like i said the way that we operate with doing that, right?

43:39Like Aura is open source. We always expect people to be able to use Aura with or without Mesmo. And so, you know, you can definitely point Aura at a different tool that's, you know, a different human in the loop process. And we publish kind of the spec there of like, if you wanted a different human in the loop process, here's how you can do it. Here's how you can define it. And then we kind of like operate on, hey, we opened that spec, we've defined that spec, but then for the actual implementation, okay, great. We're going to make the simplest path for you to realize that implementation, which ends up being just, hey, our MCP server is going to naturally provide this to you on day one, so you can consume it right off the bat.

44:15Yeah, well, and I think that's a really nice way to have a pairing between a commercial product and open source. So if you say, like, look, all the doors are open, you can do all the different things. And here's some things that we know you're probably going to want to solve a lot of times that we solve well for you. So here you go. That kind of gets into an interesting question around sort of the future of Aura and in particular around like governance, right? So is this right now, this is a Mesmo run project and you all are doing most of the work. Are you looking to get it into third-party foundation or how are you thinking about governance of Aura going forward?

44:50Yeah. So right now, Aura is a small but mighty project. And I say that because, you know, the primary contributors are from Mesmo. We're starting to see more and more people interested in the space. And the fact that we've gone with the Apache 2 license makes that a lot easier for folks to be able to contribute without having to jump through a bunch of hoops. We're a member of the AAF, so the agent foundation that's underneath the Linux Foundation, and we're active contributors there. And so, you know, there's definitely a future here where I think as we start to get more and more interest in Aura, and, you know, we have no intent of this just being like, you know, the Mesmo way to run agents, we want this to be as like general purpose of a best practice standard in the industry.

45:26And so I think like, you know, not to put the cart before the horse, because we We still want to get as much proof and partner with people as closely as possible where we can very quickly, you know, make iterations where it makes sense. But I think as it starts to mature as a project, definitely makes sense for, you know, potentially contribution to I think CNCF is probably pretty crowded. But like the great thing about AAF is it's it's it's fairly new, fairly greenfield. And, you know, there's a lot of folks that are interested in kind of moving the space forward through that. So definitely looking at that as a as a good option potentially in the future here.

45:59Yeah, totally. Well, and I think it makes a ton of sense to keep it tightly controlled to start with, right? Like I've definitely seen people try to open things up a lot too early, right? And it leads to chaos oftentimes, right? Like the benevolent dictator for life, even if the dictator is a company, like that's a good way to start a project and keep it moving. If folks have not used Aura, they are curious, they want to get started, maybe they want to try contributing or tweaking it. What is your sort of on-ramp? What do you recommend? Yeah. So one, we have a Slack community. So, you know, everybody from Esmo is there.

46:33You can kind of get help if you have an idea. We're very interested in kind of like fielding, you know, what people want Aura to look like in the future and what ideas they have in the space. And you can definitely open an issue on the GitHub repo like you can for many things. PRs are definitely welcome as well if you're interested in donating your tokens, right? Like, and we've tried to make that as coding agent friendly as possible where we're heavily leaning on like the CloudMD and AgentsMD and things like that. And so we're trying to make it as easy for folks to contribute to. But then again, that Slack community is a great place where like, if you don't know where to get started or you're having some problems or you just need a little bit of help or have some questions, you know, pop into that community and very interested in helping where we can.

47:15And we're also, you know, very interested in partnering with people. Like if you're looking at how you want to change the future of SRE and you're aligned to this conversation that we just had and, you know, you want to partner with a company like Mesmo that's kind of like looking at building this, you know, firsthand and we have engineers actively working on this, you know, definitely reach out and we're like happy to provide any kind of support or like implementation, things like that to make sure you're successful. So a lot of different paths based on, you know, just community contribution versus like true commercial partnership and design partnership there.

47:47Yeah, I love that you're doing an agent friendly open source project. I know a lot of open source maintainers have been kind of resistant to agentic coding. But having that first and figuring out how do you design your workflows and your processes and your automations and everything to make that friendly. I mean, that's to me, that feels like what makes sense in this world we're now in. Totally. Now we're in this point where the code is cheap, but the ideas are expensive, right? And so it's like the easiest thing for us to do is make that process, you know, as streamlined as possible, right? Like we know that coding agents are the future and they're here to stay.

48:25And, you know, it actually like lowers the barrier, right? There was a while where like in order to contribute to an open source project, you had to really be in depth in that project and like actually understand what's happening, maybe be an expert in Go or Rust or whatever, you know, because Aura is written in Rust. Not everybody knows Rust and we totally understand that, but every agent knows Rust, right? Like anybody who has access to Opus or any of the coding agents, right? You know, you have an expert in front of you and you have your tokens. And so, you know, now the donation to open source becomes, hey, if you want to donate your tokens, these projects like ourselves and other folks in the space, we just need to make it as easy as possible, free to do that.

49:01Well, I realized we're getting close to the end of our time here. Are there any things we haven't talked about yet that you think would be good to talk about before we wrap? You know, there's a concept that we're really keen on advancing kind of in the industry and kind of how we operate as Mesmo, which is, you know, there are two kind of like axes in the space, right? There's what production AI looks like as far as like AI in production and like all these things we're talking about, right? Like multi-agent workflows, there's how you reliably call tools, there's memory, there's things like that.

49:33And that's all like, how does an agent operate well in production? And then there's another axis, which is like AI for production, right? So like, that's where you think about like, what are the types of tasks? What are the types of skills needed? What are the types of tools needed that really enable these agents to handle these types of tasks that we see high value and something that's inherently hard to solve? And so we think about production AI and like how we're trying to shape the industry around like this is a problem. It's really the intersection of those two things where we want to enable SREs to not only own their own agents and like kind of go through this process safely and reduce the risk to get to the outcome.

50:11But like what we see as a healthy side effect, which I personally am really excited about just from being a practitioner earlier on in my career is really like this is our mechanism to also elevate the whole role of what SRE means, right? Where we're more we're architecting reliability more so than responding to things that we've implemented. Like we're taking more of a design approach for how we can make systems reliable more so than being firefighters all the time. And so I think that's a really interesting opportunity that we have to move the space forward. And like I said, I'm just, we're really excited about like where this is already starting to show success and where we see this kind of advancing in the next months and years and things like that, that really bring all the SREs today that are used to automation.

50:56They're used to, you know, handling these really high risk scenarios and just giving them the tools in the ecosystem to make that ready for the agent era. I love that. It fits really well into something I've seen in a bunch of AI spaces where a lot of people are trying to delegate their thinking to AI, but where it really shines is when you can use AI to elevate your level of thinking and really do something that's more impactful, deeper, more just leveraged, all of those different things. So can you maybe pull one example out of how using this at Mesmo has helped you elevate your SRE approach?

51:31Yeah. So one is, you know, again, like we've taken the approach of making it less like a firefighter and more like a reliability architect. So all of our SREs, right? Like we have all of our engineers on call as well, right? Like we take the approach of like the SREs are the ones building the guardrails and building the systems, building the workflows. But that makes it easier for other folks to also take that pager, right? Like be on call for the systems where, you know, all that tribal knowledge that typically was trapped in an SRE's head, like agents are a really good mechanism to make that kind of a next iteration of like as code, right?

52:05Where like we get that out of people's heads and make it more automated and make it more standard practices. And so that's really helped us to really iterate internally on what's the next iteration of SRE and really use our own ability to, you know, capitalize on our own harness and like our ability to kind of like incorporate our own learnings there. And we really just like elevated our SREs internally so much to the point where like, it's made the role really exciting and really given people internally, the ability to kind of see what the feature looks like, which then makes them also excited to talk to other SRE teams at other companies, which has been kind of a cool sight to see is like, we're getting to the point where we can actually shared knowledge, build a community, build, you know, kind of like these standard practices that extend outside of Mesmo as well.