Grafana’s Approach to AI-Native Observability

21:57Like, what are you seeing in terms of those? Are they integrated with the APIs themselves in some way too? Yeah. So for a lot of the, there is like some good integration. You think about a lot of the tool calls, right? They're not really any different to any other kind of RPC call that you're making, where you're going to want to measure, is it actually getting a response? Am I getting valid responses coming back? You're going to want to look at latency. How long are these things taking to respond? Because that's going to impact the user experience. And you talk about what does monitoring mean or what does observability mean?

22:25Why do we do it? The main reason for it is to make sure that we're meeting our customers' needs or our users' needs, meeting their expectations. So one of the ways we do that with observability and the simplest approach we have is around the concept of SLOs, right? So service level objectives, right? So you, rather than saying, hey, I'm going to go and monitor my CPU usage, right? Because do I care if I'm using 100 % CPU? Not really, right? Because I'm paying for a CPU, shouldn't I be able to use 100 % of it? What I actually care about is what is the quality of the service I'm giving my user, right?

22:57Am I seeing a latency spike, right? That's something that I care about. And so being able to kind of pick what are those kind of core, you know, user experience things that I want to make sure my application is delivering on. So, you know, reliability is a simple one, but even like latency, depending on what your business is, right, you're going to have your own objectives. And so those are the things that you're going to want to measure over time. And then as they change, that's when you know something's wrong, right? If your users are constantly getting 500 errors, where you know something's gone wrong, right?

23:22Or if you're, you know, maybe something simple where, you know, if you see traffic drop off, right, on your site, maybe that's an indication that something's wrong and you want to have someone go and look at it, or you want to have an agent go and look at it and give you an idea of what it might be. But there's, you know, ways to kind of collect all this data today where you can, you know, a lot of logs, a lot of traces, et cetera, where you can bundle this data. One of the challenges with traces, for example, one of the challenges we have with traces with, you know, this agentic model is just the volume of data.

23:46So like those conversations that you want to have usually exceed what a trace can recently contain, right, within a single span that it can send. So we're having to use kind of supplemental kind of like data stores to store that kind of information. And this is where we see then one of the nice things we'd like to be able to do in Grafana, right, is, you know, we've built this ecosystem around integration and interoperability, right? We call that big tent philosophy, right? Being able to kind of bring data together from lots of different types of data sources and be able to kind of draw those correlations.

24:14And so we're able to do the same thing. We can go and build a new database. It could be a SQL style database that's storing all this information. But within Grafana, we can just stitch that together and be able to have you kind of correlate between my traces, right? Where I've got all of my traditional kind of APM data, but then be able to also tie that to all of the chat history or all of the other kind of context information that you want to go and store and be able to then visualize that in one place. Nice. Cool. So back to what you're doing inside of Grafana itself. So you mentioned that awesome hackathon assistant thing.

24:49In terms of just day-to-day operations, where are you seeing AI have the most operational wins for you. Maybe it's even outside of observability of itself. Yeah. I mean, we're definitely very bullish on AI capabilities across the organization, right? Like we have a mandate of like everyone, you know, try and use AI wherever you can, right? So we can take advantage of it where possible. Obviously we're seeing huge adoption and use within, you know, software development, right? Like it is great for building software, for getting code out there and just being able to innovate quickly. We're actually quite fortunate because we see kind of like the role of a software engineer is changing, right?

25:26It's less about the code and it's more about, you know, engineers are becoming more product managers, right? Where it's really about kind of understanding the user problem and then being able to explain that to your agent and we'll go and build the code for you. And that's something that we're, again, fortunate with at Grafana Labs where we've always had this kind of bottoms up kind of culture where our engineering teams have been responsible for the roadmap, right? Like we have a great product team and great product managers, but they're there to facilitate or they bring the information in, from our customers.

25:53They bring information for what's happening in the industry and we can see and then come and facilitate the conversations. But at the end of the day, we like our engineers to be responsible for the product roadmap, right? Because we want them to own it. Part of that is because we are building products for engineers, right? So they do have good kind of intuition and insights needed. Yeah. And so we've always had engineering teams and had a big focus on giving them autonomy to go and make decisions about product and to think about what are the needs of our customers and how can we go and solve that.

26:20And so that's now helping a lot with the tools that we've got where they are already experienced at being, you know, kind of mini product managers themselves. And so they're able to kind of leverage the tools quite effectively to go and build things and get new products shipped and delivered. The other area we see it is where it's interesting, it's obviously more on the operational side is investigations, right? Like understanding what has gone wrong. So again, you know, I talk about the final assistant started as MVP. You know, one of the things that we like to do with our products, you know, we're scratching our own itch.

26:46And before we obviously release products to our customers, you want to make sure there's good product market fit. And the way we do that is we just make it available to our internal teams. And if they start using it without anyone telling them to, then we know it's a good product. And we saw this with the assistant, right? Soon it was in there, people started using it. They get paid for something and they just send the assistant, hey, can you go and tell me what do you think it is, right? And it's very, like, it's remarkably good at doing those investigations and going through and finding the problems.

27:11So that's been really powerful. So we leverage that a lot to go and understand. It's not always right, but often, even if it's not right, it's going to to kind of reduce the surface area that you need to go and look at, right? Because it will come with some suggestions or it might exclude some certain problems, right? So it really does accelerate our team's ability to go and find that root cause and solve those problems. The other area that we're seeing more and more is, which is a little bit scarier, right? Is, you know, as we see AI shift from just the development side into more of the deployment side, right?

27:38Like, you know, when do we start giving our agents keys to go and just, you know, you've written the code, why don't you just go and deploy it? This is where I get uncomfortable. Yeah. This is where I get uncomfortable. Yeah. I mean, it's a scary prospect, but I don't think we're far away from it. I mean, obviously we see a lot of things in the news cycles, right? With, you know, agents going and deleting people's production databases, right? So people are obviously a little bit scared about these kinds of things. But me personally, I don't think we're that far away from it, but I'll caveat that a little bit.

28:05And I think one of the things that we see is that, you know, we don't inherently trust the AI to do the right thing. That's okay. Right. We shouldn't, right? We should be kind of cautious because if there is an opportunity for it to do something wrong, it will find that opportunity eventually. But for me, I don't think that's any different to how we treat people, right? The number of times I've said to someone, oh, we don't need to worry about the normal kind of process. It's just a small change. It's not going to affect anything. I'll just deploy it in production, right? And then it's like, oh, oops, I didn't expect that was going to happen.

28:35And so we always, we need to protect ourselves from ourselves already today. And so like, you know, certainly at the final labs, we have a whole bunch of tooling in place in our CI city pipelines that put physical gates in place where it just prevents you from being able to do the wrong thing, right? You can't go and do a change where the blast radius is global, right? Like it has to be small blast radius, limited impact. So if something does go wrong, it's not going to impact all of our customers all at once. You know, we have things around cost management, right? When we do a change, we have a CI check that runs and says, this is going to increase spend by 10x, right?

29:07And you can't do that, right? And so we have got all these physical gates in place that prevent people from making mistakes. And that also works then for agents. Once you build these gates in place where they're hard blocks that stop the agent from accidentally shooting itself in the foot, you get closer to this reality where you can start to let it do some things in your environment. And I don't think we're far away from that. But I think that having those controls in place is an organization maturity thing. It's taken us a long time to build that because we add new checks in place. every time something goes wrong that we didn't expect were going to happen, right?

29:42And that's easy enough to do with people, right? Because we move much slower. The scary thing for an agent is that they can move very fast, right? So they can make a lot of mistakes very quickly. And so we're going to have to make sure we've got the right kind of controls in place. And so that's going to be the, I think, the thing that will allow us to give the agents a little bit more control to go and deploy things. You know, they can write code, they can go and deploy it, they can observe it and see is it having the desired effect? Is it working correctly? If it's not, make some more changes, go and deploy it, right?

30:09And just shorten that feedback loop so that they can base their changes based on direct feedback from users and what's happening inside the production environment. But it's a scary thing. But I also think it's exciting, right? I think it's an exciting challenge, right? Because we want to get to that place where we can free it up. Because right now, like as software engineers, you know, often our job is to balance the how do I deliver faster and balance that against how do I keep the system reliable and meeting my customer's expectations? right and so we can write a lot of code right now we can develop a lot of new features with the ai but we're still kind of bottlenecked on how quickly can we get that in production and validate that it's actually working and so i think once we we can get past that like where we can give agents a little bit more control then that whole cycle then accelerates a little bit faster and so we can then ship faster and we can get better insights and kind of keep iterating and going and it's a scary prospect and i think there's going to be a lot of pain along the way but i do think it is coming.

31:02We just have to find ways to do it in a controlled, safe as possible approach. Yeah, there's definitely a risk analysis that needs to happen there and how you step slowly into that. And if you have the right checks and balances, I can't wait for some of the stories that'll come out from this. It's going to be unbelievable. I'll give you an example of what we're doing in Grafinal Labs. So one of the things we're automating is when we see, you know, we do a deployment, right? And be small scoped into a specific set of users, right? And then we can go and look at that and make sure it works. And so we have some automated checks that we can leverage Grafana Assistant to go and say, hey, are there any problems or the new errors showing up, et cetera?

31:40And if it does find problems, we just have it automatically open a PR to rollback, right? And so those kinds of actions we're seeing kind of creep in, you know, we still have a user look at that and approve that PR before it happens, but it's simple things like, oh yeah, this is just a simple rollback, can approve that and all just roll back to the previous release, and we can then go and fix it. And so we are seeing these things where we're slowly giving the agents a little bit of operational control because they can do things much faster than we can. But we still need people in the loop right now.

32:08Yeah, self-healing system. It also depends on where that change or what change it's making, what it's affecting. Like, I don't know how I'd feel about an agent deploying like a data migration on millions and millions of rows of data. But thinking of front-end fix, something like that, something that could be easily rolled back, hey, why not? Exactly, yeah. And I think that's where we're going to start. It's those safer operations. We're going to be more comfortable, right? Giving the AI tools. But, you know, we'll definitely see there'll be a bunch of young startups where they're very risk tolerant.

32:38Like, I just care about moving fast, right? And they'll go nuts, right? They'll just let it do whatever it wants and some databases will get deleted. But that's okay. Hopefully not important ones. So let's talk about the people using these tools, right? Right. So as AI handles more of this operational work, how do you see the role of SRE, DevOps engineer, platforms engineer changing? Do you think they're getting bigger, smaller? I mean, I think right now they're bigger, right? Just simply because the volume of new software getting deployed is just growing so quickly, right? That said, I think they're getting left with, which is actually kind of, well, for me personally, right?

33:16Like they're getting left with the harder problems, right? So a lot of the toil, the simpler work that the SREs used to do can be done by the AI now and be done by the agents now, which then leaves the SREs to focus on more high value kind of pieces of information. So the hard problems, right? How do I go and diagnose this or, you know, thinking more strategically around like, how do I build a more reliable way of doing things, right? More kind of process orientated or, you know, go and find, you know, how that AI is not supporting the needs of the business or, you know, helping you kind of drive things forward.

33:44So right now, I think there's just growing need for it, right? We're shipping a lot of code. It's of questionable quality sometimes. So we just, we do need more of that kind of SRE kind of practices to, you know, make sure we are thinking about reliability, make sure we understand, can we measure it? Are we making sure that, you know, as we're building cool features and new tech, is it delivering the value that we want it to have, right? And meeting our customers' expectations. And most importantly, when things go wrong, do we have the data to be able to understand what happened, right? What went wrong so we can go and fix it.

34:12So I think that's definitely the case. As that changes over time, I think, yeah, we're still going to need those practices. The thing that scares me the most, both from an SRE perspective is also software development. It's just that because all of the easy things, right, are being kind of taken care of, that's kind of the work that you give your junior people, right, or new grads, where you just have to go and do it. And you kind of, as you're doing it, you learn, right? You learn how these systems work, you learn what to do. But as we take that opportunity away, it's like, how are we going to give people the opportunity to develop the skills that they need to become these experienced kind of senior SREs that we're still going to need, but they're not going to have that path, right?

34:48And so that's the thing that scares me a little bit is what's going to happen in 10 years time when we realize, oh my God, we just don't have anyone who has the skills that we need. And so I don't know how we fix that. I think it's the same with software development, right? Like we still need real experienced software engineers, but the new grads and kind of juniors aren't getting the opportunity to develop their skills where they can have that experience, right? Where they've learned things the hard way and have gained that experience, right? They've earned it by making a lot of mistakes along the way.

35:15But, you know, we're taking that opportunity away from people. And I think that's the scariest thing for me when we think about the future. The high-level thinking that's needed just keeps getting more and more and more. And it goes from reactive and detail-oriented things to proactive high-level thinking. And yeah, I'm in the software engineering game. We're seeing the same thing. I saw an interesting, I wish I knew who the author was, but I saw an interesting thing of one potential way out of this is to go back to like a apprentice journeyman master model, which I am a huge fan of software as craft.

35:49And I thought that that was really interesting where it's like the only way that you are going to get this experience is by sitting shotgun to an experienced person and feeling the pain with them and seeing their line of thinking. I think that's especially interesting with SREs because you're reacting to these potentially nasty problems and trying to unravel that. And so I thought that was a really interesting one. I think it is an approach. I think the scary challenge is like an expensive approach where you're not getting immediate value from, right? And I think a lot of businesses and organizations are going to be like, I mean, sure.

36:27Why don't we just let some other company be the ones that train all the people and we'll just keep using the agents, right? So I think it's going to be very difficult to find that. I think one area where we do see this opportunity is still open source, right? Like we, you know, we'd love to kind of engage with our community. We do get a lot of young people who come and contribute and that's our opportunity to kind of coach them, right? And being open source, right? We're happy to invest that time, right? To nurture those people so that we can, you know, help them develop their skills over time.

36:52So I will see there will be more of those like community driven, right? Kind of inconsistent. Because I do think this idea of training people up to be productive so that they can leave and go get a job at another company, right? It's not going to be an attractive option for a lot of organizations. That's a really good point. So I do think more of that, like as software engineers, as a community, I think it's our opportunity to find ways to kind of nurture those people coming through and give them opportunities. I think open source is a great way to do that, right? There's big code bases, right?

37:19There's opportunity to write code. There's ways we can kind of communicate and share ideas and help kind of train and coach people a little bit to give them the experience that they're going to need. Well, also, not all apprentices in the trades are paid either, right? And so we've been spoiled in this tech industry, but high paid internships that lead you to a high paying job, the playing field may be leveled, but yeah, good points. Yeah. But I think even like a lot of the people I talk to, you know, colleagues I've got, you know, even still people we hire today, like I love to go to, we do our onboarding, you know, in person and get people out to go and talk to the new starters of what's happening.

37:53And a lot of them have similar stories, backgrounds to me, right? Where they're very self-taught, right? And I think that's true within this industry where a lot of people love the technology They're excited about it. They love the fact that things are changing all the time and there's always new things to do. And so you have to have this mindset of wanting to go and teach yourself and learn new skills. And so I think that's still going to happen, right? But I think open source is a great avenue for that, where it's an opportunity for you to go and do something off your own bat to develop those skills that you're going to need in the future.

38:21Because the speed at which things are changing in this industry, right? You have to be committed to a life of learning because you blink and there's a new JavaScript framework. So I hadn't planned on talking about this, but I feel like it could be fertile ground. So are you seeing open source projects struggle with the amount of code that agents are producing? Or what are you seeing in your area? A little bit, both from the volume as well as the quality. Yeah. So it's very easy to have an agent just go and build a feature, right? But just because it works doesn't mean it's code that you want to maintain over time.

38:56And so that's, I think, where we're seeing the challenges. is I think this is still where there is, you know, that challenge with the AI assisted kind of code tools where we put a lot of thought into kind of like the architectural design and guiding principles for how we build code, right? And every organization is different, but you end up building your view of the world and how you want code to look, right? So it's got consistency. And so it's much easier for your teams to kind of like people to move around between your teams, right? Because the code bases are all a similar kind of layout. You're using similar kind of practices for how you go and build code.

39:26And so things look a little bit more consistent. Whereas when you start getting the AI in, it doesn't care about those kinds of things, right? It's just going to find a solution that works and go and throw that code in. And so there's a lot more work that has to go on to think about and care about maintainability, right? So that certainly affects open source, right? Where you don't just want to accept any pull requests, right? That adds any new feature, right? You want to think about, is that feature right for this project, right? Is that something that we can maintain over time? How does that impact kind of the rest of the ecosystem?

39:55And so that's where we're seeing the challenge is that, you know, you'll get people that will just submit a pull request, you know, that'll be 20 ,000 lines of code that, you know, the AI is generated for them. And you're like, well, no, right? We're not going to review this. It's too big. It's too much. You know, this is not how we build software. You need to kind of break this into smaller chunks. And so that it can be reviewed and so it can be maintained over time and understood. And so I think that's the challenge that we're seeing is code is a collaborative business, right? Especially for open source projects.

40:23And so the agents have to fit into that collaboration mindset of not trying to overwhelm people with too much change all at once because it's just not going to get accepted. It's too hard to review. It's too hard to maintain over time. And to your earlier point, these things might be learning moments for people that are less senior too. Yeah, definitely. So let's move towards the future a little bit. So what's the operational problem that you think AI and these tools are going to crack in the next two or three years that people are still doing manually right now, even right now with the existing tools that we have?

40:56Yeah, I mean, I think right now we're at the stage where AI can be that first responder, right? When something goes wrong in your environment, right? So an alert gets fired because, you know, you've got an SLO burn. It's like, hey, you know, error rates have spiked. Something's wrong. You're going to page your on-call person, but at the same time, you can go and spin off, certainly in Grafana Cloud, you can go and spin off in investigation. So, hey, go and try and work out what's wrong, right? By the time your on-call engineer gets in front of their PC, there's probably already a response to say, hey, this is maybe what the problem is, right?

41:26It may have worked it out or it may have excluded a whole bunch of things. So having that, I think, is where we're going to see over the next few years. And that's just going to get more reliable, more accurate over time. We're going to build more playbooks. We're going to build more skills for it so that can do more interesting things. But I think that's where we're seeing it is that first responder, as soon as you notice something wrong in the environment, it will go in and it will try and troubleshoot that and find that root cause for you. So that way, instead of it taking an hour, right, to find that root cause and kind of solve the problem, we're now talking in 10 minutes, right?

41:55We can go and get things patched up and move on. We're still going to have the follow-up process, right? Which, you know, is so important for our SREs is to go and actually don't just stem the bleeding, let's go and fix the problem. So that's still going to be important. but again you can leverage your AI tools to kind of process more data and kind of do those kind of things but I think the near term so the next couple of years that will just be standard right like you'll just there'll just be an expectation that 60-70 % of the problems are kind of resolved before your on-call person even gets in front of a computer yeah because right now for resiliency in systems we rely on redundancy and things like that but I think what you're saying is what we're starting to trend towards is actually in a limited sense, but a self-healing system, a dynamic self-healing system, which is pretty amazing.

42:41Yeah. And I think that's going to be really important just because of the speed at which software is being shipped, right? If you want to move fast, right, you're going to inevitably break sooner, right? And more frequently, right? Because you're trading speed for accuracy, right? And so the need for better observability just becomes more apparent, right? You're going to want to make sure you've got the data that you need so it can go and fix. But also we're just seeing that AIs are being able to use that observability data to understand what went wrong. And that feedback can be really fast. And that's just going to help us be able to ship things faster, but also maintain the level of reliability that our customers expect from us.

43:16And I think the other thing that we're seeing AI very good at is obviously processing huge volumes of data, right? Going and looking for those kind of trends, those hidden things that a user hasn't noticed. We do a lot of that internally where we can just say, hey, just go and query a huge volume of logs and find what's interesting. So we're seeing that use case change. We recently made changes to Loki, so a log aggregation system to just be able to do that smarter and better, right? To fit with that use case of what AI agents want to go and do. So we've made a whole bunch of architectural changes and performance improvements so that it can query 10 times faster.

43:49More importantly, when you give it a query, it can reduce the volume of data that it needs to go and scan and look at to go and do that. So that's - I'm curious at a high level what those architecture changes look like. That sounds like a very interesting problem. Yeah, I mean, so one of the things - You can't talk about it. I can, yeah, yeah. We announced it last week, Acrofonicon. So it's in open source, right? So yeah, I can definitely talk about it. I mean, I'm allowed to talk about it, whether I understand it as well as the people who have written it, that's a different question. So what we really think about is, you know, Loki was designed for cost efficiency, right?

44:22So how do we store huge volumes of logs as cheaply as possible? And the way we did that is just don't index them. The use case we designed for was the developer use case where you just want to kind of grep, basically through your logs. Grep-CI. Look at the log message. I'm like, it's not that message. You just keep excluding things. Or you might have a, oh, I'm looking for a certain string within the log mission. And so that's what we kind of optimize it for. But more and more, we see people wanting to do more kind of analytic style queries or needle in a haystack queries more often where they're like, hey, I've got a transaction ID and I knew it happened sometime over the last four days, go and find it everywhere in their logs.

44:57And so Loki was designed, you know, to be cheap to ingest data, right? But we wear the cost on the query side, right? Where we just brute force it. We'll just go and look at all the data and look what it's trying to do. And it can do that quite efficiently, but it still takes time, right? When you're talking about terabytes and terabytes of data that you want to go and process every second. So one of the big changes we've made is we kind of introduced kind of similar to light bloom filters, but it's a slightly different kind of approach where we can then know which portions of the data to go and look at and which ones we don't need to worry about.

45:26So if you're doing that needle in the Haystack query, we could very quickly say, hey, actually, you don't need to go and look at the petabyte of data we've got. You actually only need to look at these 10 terabytes that we've got here. And it can do that very, very fast. So that's one of the ways. The other way is we've just put a lot of effort into the query engine itself, just to optimize it and just make it faster, being able to paralyze it as much as possible, because that's easy to do, right? With compute systems today, right? Where we, you know, we've got a huge volume of compute where we can just go and spread out the workload and get things done as fast as possible.

45:56That works insanely well for us as a cloud provider, right? Where we've got the benefits of kind of statistical multiplexing, right? Where we can, you know, we've got thousands of users all using it, right? We can provision a very large infrastructure estate and, you know, not every user's querying it all at once, right? So we can say, hey, this one user's querying it, this second they can go and use a thousand cores. And then the next second, a different user uses a thousand calls. And so we can get really fast response times because we've got a bigger environment that we're able to kind of build on top of.

46:22Efficiency is a scale. Yeah. Exactly. Yeah. Man, I would love to just talk about that for an hour. That sounds interesting. All right. So a couple of wrap up questions. This one I'm really curious about. So what's an operational scenario that keeps you up at night, especially related to AI, something that you think the industry isn't taking seriously enough right now? Well, that's a good one. I think one of the challenges for me is just, you know, a lot of the AI agents that are being built today are just black boxes, right? And we don't know what they're doing. Sometimes we don't want to know.

46:56So I think that's the scary thing is the speed at which these systems can operate and make changes and do things, right? You know, because we've moved from the chat, right, to this agentic model, right, where it's like, let the AI do things, right? And not just respond to questions. So that scares me, right? When you've got these black boxes that are just, no one knows what they're actually doing, right we have a mandate you know internally where our engineering teams are leveraging AI wherever they came to build software right but at the end of the day the person who merges that they're responsible for that right like they're accountable for that and so I get scared about when that starts to erode right and it's like well who's actually accountable for the things that are getting deployed into our production I was having a conversation someone actually just yesterday right talking about kind of like this governance problem right we're seeing more and more agent to agent communication, right?

47:41What happens when you've got an agent in your organization talking to an agent in another person's organization and they're just doing things, right? And something goes wrong. Who's accountable for that, right? Whose fault is that? Who's responsible for kind of remediating that and take accountability? So that's the thing that scares me the most is around, well, I guess two things. One is the black box aspect of it of like, what are these things actually doing? And the other one is the accountability. Who's responsible when things go wrong if we are giving all this autonomy to our AI agents. Some really interesting legal and contractual things that'll come out of that as well.

48:13Yeah, that's a really good point. It always comes back to humans anyway, right? What's the throat to choke if something goes wrong? Yeah, that'll be the most lucrative career in the future, right? Where it's just AI, you know, scapegoat, right? And cleaning up. Yeah, I'll take responsibility. going. Blame me. AI insurance. Ooh, okay. Hey, you want to make a new startup? No, I'm just kidding. So, all right, let's close with, I always like to leave the audience with something. So, if you could give one piece of advice to our audience about how to evolve their operations for an AI first world, what would it be?

48:49Yeah, definitely. My strong opinion about this is don't go and build an AI solution, make AI part of the solution, right? So, we've got so many great products out there. You don't need to go and write a new product, right? That's AI first or something. Really where we see the value in AI is when it's behind the scenes, right? When it's not in your face, there's just, things just seem to magically work, right? And you don't think about it and you don't know why. You just know that it works, right? And you got the response that you want. As we think about AI, I think that's the approach that we want to have is leverage its great capabilities, but it doesn't need to be the only thing, right?

49:25That you're doing, right? I was a new and shiny thing and everyone's like great it's now internet enabled right now we don't care right it's just an expectation that things should just work and it happens to work because it's leveraging the internet and the technology that we have i think ai is we're in that same face right now we're in that hype cycle everyone wants to label everything with oh it's ai it's ai it's ai we're really users are going to care less and less of that over the time what they actually just care about is does it work right is it doing the thing that i want it to do and does it make So like, I think that's the focus is just making sure that your AI kind of works with existing kind of workflows.

50:01It's coming to meet people where they are and help kind of accelerate what they're doing and not trying to force them to have to go and do things they don't want to do. Well said. Well said. Well, thank you very much. Really appreciate your time. No worries. Thank you. Learned some very interesting things. Thanks for being here. Thanks a lot, Matt. It's been a pleasure.

50:25Thank you.