Unlocking the Data Layer for Agentic AI with Simba Khadder

24:49So today we have agent memory server, which is open source. And how it works is you can define different types of, I forget what they call them, but I always think of them as transformations, but they're actually prompts. So you're defining strings of what you wanted to accomplish or the type of like, hey, I want you to keep track of decisions that were made. I want you to think of seasonality concepts. I want you to think of grabbing facts. I want you to be optimized for personalizing for users. So there's different types of transformations you can write and you can write custom ones. I do think it's really interesting to start.

25:21A funny problem with building agents is a lot of times you try to stack agents on top of each other to solve problems. A good example of this is like LLM as a judge. It's like, okay, well, we have this problem where like we can't tell if these are good responses and there's too many of them for human screen. So why don't we have an agent go and do it? But how about the LLM judge is bad? Could we like write an agent on top of the LLM judge to judge the judge so it gets even better? And then you can kind of you just like keep taking the derivative forever and put agents on top. I do think that we will have self-healing memory in a way where even the memory engine gets better over time.

25:58But I think where we're at today is that you can just control the memory engine. And this is part of building a context engine, which is that you tuning those knobs. But yeah, for sure. I mean, the layers of abstraction are getting higher. We're already seeing like, I find myself having to orchestrate, like when I'm building, I'll be orchestrating like 15 agents and they'll be running. And then I use Codex today and they ping, they literally like make a sound, which is kind of annoying because I feel like I'm a waiter. I feel like I'm running from like table to table to go like wait on them. Well, we could talk about that because I think there's some meat there too.

26:29So the second question related to this is around controllability, because I think one of my personal frustrations with most of the memory systems that I've used to date is they treat all memories the same in the sense that like I'll use ChatGPT as an example. chat GPT, they added memory. I've had many conversations with it. And now I'm asking for perspective on this thing. And it's referencing some completely unrelated concepts that I talked about three weeks ago. Like I have a lot of different interests. I would like to scope your memories, please. So don't tell me what you think I wanted because I previously talked about cooking, right?

27:07Don't make a cooking metaphor when I'm asking you about code, just talk about code or vice versa. So I'm kind of curious how you all think about the control knobs for users on memory. Yeah, actually, my story, which is funny, was I used it when I was planning my wedding. I was using it as anyone would for all the random details. How should I do this? And how should I do that? And whatever. And to this day, it still is like, since your wedding is coming up. I know, right? You're like, no, the LLMs are terrible with time. Yeah, they're terrible with time. They can't deal with it. You're like, I want this to be scoped.

27:43Keep it closed. So there are a lot of concepts we have and are tunable around memories being able to fade away temporally. Like if a new memory comes in that contradicts an old memory, then the new one should be used. You also talked a bit about like having separate categories and subcategories and almost like layering these things up. Some of this stuff exists already in the memory server. Some of this stuff is like clearly where we're going. There's always UX things, right? Because if you start adding those layers of like, okay, well, there's, let's say team wide memory and an individual memory and then there's org memory and it's like okay well how do you choose which one goes where what happens if the agent remembers something that's really personal and puts at the org level and now everyone knows that you love katie perry and it's those sorts of things that really make it tricky these are problems that we have super smart people working on constantly well nobody gets it right right spotify thinks i love katie Perry, because my 10 year old does.

28:38Exactly. The thing that is funny is like, I was in feature stores before. I feel like I was kind of broadening distributed systems before that. I think I've always found myself gravitated towards problems that just don't have a solution. There is no right answer. And that's fun because you can keep pulling at it and you can keep learning more and you can be creative. And I think that's where we're at with memory is like at Redis, we're being really creative and creating unique solutions for it. So on the subject of problems where there's no good solutions. Let's go back to your point about orchestrating 15 agents.

29:07Cause I feel like this is what every, not everybody, most of us are trying to do is like we're orchestrating all these different things and they're going in parallel. And I don't know about you, but it fries me. Like I get too many of these things going. And then by like three o 'clock in the afternoon, I'm like, all right, I'm done. I have no brain left. What's funny is I love it. It actually really fits into how my brain works. And I see it really frustrate one of my, He's like one of our best engineers. But some of the things that made him one of our best engineers are actually getting... It's like he's learning how to pivot those things into this new style.

29:43Because the thing he's good at is he could like kind of single thread a problem really well. Like he could give the hardest problem in the world to him and he'd be able to like single thread it. But now it's like, yeah, but I still need you to do that sometimes. But a lot of what I need to do is actually just do that in eight things at once and jump between them and context switch. And as a founder, that's all I did all day. So I feel like I've been training myself for this day, not knowing it. But people who have not had similar experiences. It's actually interesting because people who've been PMs, people who've managed engineers, they actually tend to do better with agents in this problem than people who haven't.

30:21And it's because in some ways, there are lots of skills you got to learn to manage engineers, which are really similar here. You have to context switch constantly. As much as you'd like, sometimes you can't sit down and take the keyboard and just go do it. So you have to learn how to like help build a system or like kind of prompt your way into getting the results you want. Obviously, it's different, but there are skills that are transferable. So the short is like, I think everyone deals with it differently. I think that it's fundamentally changing how we do things, even at Redis on the org level of how we build products.

30:54And I think it's only going to accelerate. Can I ask you a little bit about those changes? Because certainly having conversations, and it's interesting, right? Because I have lots of conversations with people who are pushing the bleeding edge. And then I'll talk with somebody, maybe not on a podcast, and they'll be like, is that really how people are doing? Am I so behind? Am I there? So I think it's really helpful to have real life examples of how is this changing what you're doing at Redis? You're not at OpenAI, you're not at one of these bleeding edge, but you are adopting and changing everything.

31:26So how is it working? So I think there's a lot of aspects to it. One aspect to it is actually, it's funny, because it's actually one of our predictions for the year. And then it just happened naturally internally. And we're definitely seeing the results, which is that everyone will become a coder. Everyone can build. When I work with PMs, I don't want to read a PRD. I'm like, you could take this PRD and turn it into a prototype in the same amount of time. Let's do that. There are times when a PRD makes complete sense, but you can go build a prototype. And the other thing is features. Like it's very easy now.

31:59This wasn't a problem as much before where you can build these like, you get feature happy. You're like, oh, I'll just add this. It's just one prompt. I'll just add this thing. I'll add this thing. And so taste is really important. Like what are we trying to do here? What does good feel like? And good is not, oh, you have every single knob in the world because you can do that now. It all works together really nicely. And so the people who do really well in it are able to move that fast because fundamentally, like you can just move so much faster now. You can think of the user, could be a literal customer, it could be an internal user, but you're thinking from their perspective, what does good feel like?

32:36And you're just constantly like, where can I use AI to make this better? A good example of this is like, it's actually a story from my wife where she had all these spreadsheets. She does marketing. She was trying to figure out events to focus on. And before she would like try to scan them or just try to figure out how to use Excel in a way that was good enough to be able to get an answer. And she was like, wait, I have clogged. So she like found and built this actually a website. She doesn't know how to write a line of code. She's never in a line of code before. She built an entire website and shared over the marketing team where could click into things and it had graphics.

33:14And I was like, wait, you built this? Like it blew my mind that that was possible. So I think everyone will become one. And I think that's one thing we're seeing is that it's not just an edge thing. So that's one aspect, which is that every single everyone should be using these tools. And the thing is, is actually most of the value is going to come from the engineers. I think a lot of them get it, the different layers, there's a lot to learn, but give someone who's always has all these ideas and they've always want to build something or they're like, oh man, like if only I just had an engineer assigned to me so I could have them build this like one-off thing.

33:47Now we can just go do it. And so that's where we're seeing a ton of change. That's one. Two, on the end side, I think focusing in terms of like our team, we write a lot of specs. We review those specs. We actually find the architecture, both on the software level and actually on every level, is the thing that matters the most. If we can define an interface and you can define acceptance criteria for that interface, I'm like 98 % confident that the agent will be able to build something that solves for that interface with the acceptance criteria. If it's wrong, because it wasn't clear, the interface wasn't good, or the acceptance criteria were missing things, in which case, that's a design problem.

34:25And so anyway, I could go way deeper, but those are some things that are top of mind. Yeah, it tracks with what I'm seeing as well. I think the non-technical example is is kind of interesting because it does mean, you know, I think for core product, the temptation when you can build everything faster is to build everything. And as you highlight, that's not necessarily actually a better experience. But when it comes to like tools for yourself, it's phenomenal. And to go back to your like customer service agent or something like that, maybe you have a real customer service person, but they're building themselves an agent to help them do their work.

34:59So kind of curious, like, are there ways in which the context engine that you want to expose to people should be different for those non-technical use cases, right? We talked a lot about, okay, we're using this, we're building a product or we're building a customer service agent that's going to run autonomously. But is there an internal tools context engine? What would that look like? How does that vary? A lot of our internal Slack bots are, especially the new ones coming out, are all built on top of context engines. We had one example of a bot someone, actually this is a personal bot someone built for themselves.

35:30They would ask questions about their calendar. on their Google Cal. And they'd have all these API calls that they hadn't made. And they'd ask a question like, what are my most important meetings this week? And like, oh, when did I meet with this person last? And they'd ask these questions and it would take 20 minutes because the response time and Google API is not made for the sets of queries you'd ask. So then they actually had the agent build a sync thing so that every time their calendar changes within a few seconds, it just syncs it into Redis. And then they put a contact surface on top and they were able to connect to it that way.

36:06So that's one example. I think for internal apps, I think we will see something. It's going to look different because a lot of what Redis style is today is much more oriented towards people building with some sort of skill in mind beyond like a single user. I think that you can use it and a lot of people are using it in like kind of one-off. Oh, I have my own Redis database. And that is my, just because of the data structures are so nice, because it's so easy to reason about, because the interface, you can just go write commands. Like you can go write the wire format in English, you know? It makes a lot of things easier and really fast.

36:39So I think people are doing it that way, but I do think it will look different. I think the thing that will be really unique and things that we are thinking about is like, if everything looks like a coding agent, like OpenClaw is essentially a coding agent. That was the whole idea. It's like, if we give a WhatsApp interface to a coding agent, it's essentially a generic general purpose agent. And really then you start thinking, okay, well, what do we need? We need workers, cloud workers. We need a cloud file system. We need the ability to pull data from different places and make that data accessible.

37:10So there's something there, but I think that it's still early days. I'm really, I get super excited just thinking about what's possible. Yeah. If we can solve like the data sandboxing data access problem, because like big challenge with open clause, it's like, okay, to put this on your box, go. Now it has access to everything. But like an agent that looks like I can write code, I can run code is a general purpose agent. It can do anything. And if we solve the data layer around that so that that can be safe and somebody doesn't have to think too much about it, like that's beautiful. So with that concept in mind, then are there particular frameworks for building coding agents that you see going forward?

37:46Or like, you know, if everybody's going to build their own personalized coding agent, what are the tools they're going to use to do that? So on the agent framework side, I think the ones that are going to be most successful are the ones that work best at the ecosystem. I think that's really what it comes down to, because I think what we're finding is that to make these agents successful, they need to interface with the world. And agent frameworks that make it easier to either extensively do that or have built in things to do that, make them much more powerful. For example, this isn't true. I'll use a contrived example to show something that would not succeed.

38:22Let's say someone's like, ah, MCP sucks. We're just not going to support it. We're going to build our own version of MCP. They might be able to build something that whatever is more efficient or better in some way, but it doesn't matter. The thing that's like unique about MCP and valuable MCP is the standardization. Everyone's doing it. And so what matters more right now is standards, because if we can build so fast, the only thing you want to avoid is having these proprietary blocks in front of everything. Make everything speak standards and make it, it doesn't have to be standards, it's like make it clear how to contractually touch all the things, and then I can think of and build any agent I want.

38:56So in terms of the frameworks themselves, I think the ones that are most successful, most extensible will be most successful. And we see LandGraph a lot. Obviously, super powerful, very early player in the space. They have this DB agents framework, which I think is really interesting. We see ADK, obviously a lot, like a lot of the cloud providers are releasing their own solutions. And ADK is really good. I like ADK a lot, actually. There's like some kind of more off ones, but I like one called Agno, for example. That's another agent framework. But in short, it matters, but I think it matters less than...

39:30I think as engineers, we get so caught up in our tools. I talk to people who, with our coding agent setup, they have this crazy setup that they put together. I'm like, yeah, man, I just have like 100 Tmux windows. I switch between them in this way. I have the most simple ad hoc setup because it's what works for me. I don't need any... In fact, I didn't use Cursor for way too long because I didn't want to switch to an IDE. I was like, I use Vim. I was so happy when the CLI-based versions caught up because yeah, I was the same way. I'm a longtime Vim user and cursor, like you had to use it because it was at the bleeding edge.

40:06It was the best. And even though it sucked, it was still the best, but it's not the best anymore. And so I'm happy back down in my also Tmux enabled, like whatever, I have a bunch of work trees and a bunch of Tmux sessions. I'm like, go. Same. And it's funny because if you had told me, I also haven't ran the line of code in like maybe almost a year now, which is crazy for me to even like imagine. But why would I? It's lower leverage. What's funny is I also feel like I've become a better engineer at the same time because I'm no longer caught up. I feel like the skills of an engineer was like almost can you Google fool your way into some weird random library to solve this weird random error or kind of like dig deep through the internals of a code base and find this random bug.

40:48And I almost feel like that's not that important anymore because the agent can go do that. The thing the agent's really bad at is maintaining a consistent architecture that scales well and makes sense at the code base over time, especially with the backdrop of I can put out 100 ,000 lines of code in a week by myself, times that by team size of Redis. And it's actually easy to almost overwhelm the system with productivity. It's not meant to handle that. And so the issues actually move downstream. And this comes back to this question of like, how are you all adapting, right? Because once again, like reviewing 100 ,000 lines of code in a week times N engineers is incredibly hard.

41:29So now your reviews maybe have to move upstream to the design or something like that. I don't know. How are you all handling review and the feedback loop there? So there's a few pieces of this. One, it depends on the code base, actually. I think different languages and different severities, like an internal tool, you can get away with a lot more. And Redis core is different. It's a different beast. That said, the thing that makes the way I think is one, like we will do design sessions. We do them every day. I don't like standups and we just do it async, but the design sessions, I like doing sync where it's like every day someone comes up for like, this is the system I'm building.

42:06This is this big, the project I'm working on. They'll show me the interfaces. They'll show me how it's all going to work together. I make sure that we really deeply understand what we're going to do and what the acceptance criteria is for all of it. if that's good, then I'm like, yeah, then just go manage some agents to do it. As long as we all agree on what we're doing and it makes sense, then your job kind of becomes, okay, now go manage agents, go do it. So the first piece is the engineering rigor is much higher. Another example of this is, and it used to be where someone would make a PR and it's not exactly what I want.

42:34It's not bad, but it's not what I would have done in. I don't really like it. And I may even tell them and they might be like, yeah, you're right. It's like, do you want me to go rewrite this? and you're just like, we have so much to do. Like, this is fine. And that doesn't happen anymore. Now I'm like, I mean, the code's cheap. Throw it all away and change the spec and regenerate it, you know? And if I keep it, say that you want the same behavior, but you want it to work this way and then figure out the path to get there. So that's one. Two is tests. Behavior tests are like the most important thing.

43:03A lot of times when I start, I will start with what is the golden path, both on the error cases and the happy cases that would show with high certainty but this thing is going to do what it's supposed to do end to end. So I don't, I write a lot of end to end tests, but those are the ones I manually look at and make sure those are good on every PR. I'm like, okay, what do we know works? Did we miss anything? And then obviously I still expect test coverage to be high. I expect a lot of integration tests, a lot of unit tests, a lot of fuzzing on parameters, stuff like that, obviously. But I think the flow has changed where I almost would go as far as to say, if the behavior tests are correct and they pass, then the code is right.

43:44And if you're like, well, how about the speed? And how about this? And it's like, well, did you have a behavior test for it? If those are important, they should be captured in the behavioral tests, right? Yeah. They should be captured. Yeah. Because then it's doing exactly what's designed to do. So all that matters is the behavior tests. So I'm with you, but I'm going to bring back to a slightly different thing, which you talked about, like the really challenging thing, the really high engineering thing here is how do you keep a system that is architecturally coherent and continues to be maintainable and evolvable over time?

44:13So I have seen agents build things that completely match my behavioral tests, and yet create all sorts of like entangling of concerns that are going to make it very hard to do things. So how do you square that piece of the challenge of the code base? Yeah, so it's almost like there's three steps. It's like step one, which is the architecture review. And we do that synchronously. The other thing that really benefits, which a lot people don't talk about, is really good engineers who embrace this fully are like ridiculously productive right now. But there are a lot of people who are trying. I see them, they're embracing it, but they're not able to hit the same layer.

44:50I didn't even realize until I saw how big the separation can get. So a lot of the design sessions are one, to make sure that everyone's looked at the architecture, it makes sense that we didn't miss anything. Two is it kind of works is a knowledge osmosis, which is really important. But anyway, the first piece is architecture. So I'm assuming by the time we got to the behavior test that we've already decided on the architecture and I'm expecting to see the same interfaces that we talked about and that the spec, which we have written and looked at, which is by the way, in plain English and not super long, it's long enough, but it's not.

45:19A lot of people end up implementing the thing in the spec, which is not the point. The spec should be readable by human and reviewable by human. And then what's funny is like, I used to still treat code review at the same rigor I used to myself. And then what I would find is that BugBot and similar products would actually find things that I missed. And then it just kept happening. And I, you know, I spent a lot of time code reviewing it. Some of the things that finds like, oh, there's this race condition. And I looked at how it figured out what the race condition was. And I was just like, I give up.

45:54I'm like, there's no world I would have been able to put all those things together to catch this race, which by the way, would like never actually be triggered. But it's there. Where is this like crazy? It is real. Yeah. No, I agree. Bugbot in particular is shockingly good. I don't use the cursor ID anymore. I would not survive without Bugbot. It's phenomenal. But yeah, so I think that is really interesting. And I think one of the things that is really important there that you're calling out is the fact that these things are doing the code has not lowered the requirement on engineering. If in anything, it's increased it.

46:28And you're doing these very synchronous discussions and debates. And you slow down in some areas in order to be able to speed up and just generate code quickly. Yeah. The trick is how do you do that and not end up in a waterfall-like system that some people push back on. And the idea is that we come back and edit the specs. We don't always get them right. And sometimes going through it, you actually, you can get better. You can iterate on them. A lot of times you can't even like the size of the work done is not one thing. It would take a few things. So we kind of break them up into almost like sub specs and there's a whole process there.

46:58But the cool thing is for us, like I read this is like, since we've gone so good at this and we've embraced it fully, like one thing that we do in our leadership does, which is really amazing here is that like the default answer is yes of AI in a sense of like, we try and enable everyone. Like I talked to a company where it's like, yeah, we're just getting access. Like some people have select access to like, you know, this. And I'm like, you don't even realize how behind you are if you're not letting your engineers use this stuff. I understand, you know, why people might feel that, oh, you know, we need to control this more.

47:29But it's night and day. So it lets us build amazing products really quickly. And because we've learned how to do it early on and we haven't let the quality drop, that's other thing. is there's a difference between agentic powered engineering and vibe coding and nothing wrong with vibe coding, but I mean, all my demos are vibe coded, but that's their demos. I don't vibe code database code. Yeah, there's pace layers, right? There's like, okay, this thing can be vibed. And if you don't like it or it breaks, throw it away, do it again. This thing, you gotta keep it working right. Yeah. Awesome. I think that's super helpful.

48:03We've covered a lot. We're getting close to the end of our time here. Is there anything we haven't talked about that you think we should discuss before we wrap? I think the big thing and the biggest takeaway is in this next phase, which nowadays that could be like another year or two, but the big change is going to be the switch of systems from linear rag style to context-based. So everyone's going to be building these context engines. And I think that that's just true. I mean, we at Redis are obviously enabling it and building a ton of awesome products to enable it. But I think that that is the biggest shift because that attention is all that matters paper is what kind of got us here.

48:41And I think if you're building real products, you just assume that someone else is dealing with attention and for you, context is all that matters. Context is all that matters. Love it.