Rebooting Enterprise AI with MCP and Kubernetes

0:01Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at PracticalAI.fm. Now, on to the show.

0:41Welcome to another episode of the Practical AI Podcast. This is Daniel Leitnack. I am CEO of Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer. How are you doing, Chris? You're doing great today, Daniel. How's it going? It's going really good. I feel like I'm going to just add a bunch of tools to my toolkit today via MCP, because today we have with us Craig McClucky, who is CEO of StackLock, which we'll learn a little bit more about and of course talk a good bit about MCP and other things. Welcome, Craig. It's great to have you.

1:25Hey, thanks for having me on the show. Yeah. Well, I kind of, I mean, and people can look up the projects that you're working on now on StackLock. A lot of that has to do with MCP and AI on top of Kubernetes. Do you want to just give us a little bit of a kind of setup of why you are spending your time thinking about this intersection of AI, MCP, Kubernetes? Yeah, I mean, I think for me, it's sort of interesting. And maybe I can just introduce my background a little bit. I'm an infrastructure guy. I've been building infrastructure technology for pretty much the totality of my career. I built a lot of infrastructure tech when I was at Microsoft.

2:07And then at Google, I was happy to meet my friend Joe and we built out a Google Compute Engine, which is, you know, virtual machine infrastructure technology. We started the Kubernetes project, did a few other things, you know, inside there. Subsequently, we've built a lot of other technologies together, you know, either as in the startup context or as part of larger companies. You know, I think for me, there's this sort of a lot of interesting parallels. I think, you know, history doesn't repeat itself, but it often rhymes. And, you know, one of the things that got me really kind of captivated by this whole MTP thing was, taking me back to my earlier career, I remember the first time I saw Docker.

2:44And so Docker is a technology, if the audience isn't familiar with it, that enables people to package up an application and all of its dependencies in a single container that's then highly portable and can be deployed everywhere. And when I saw Docker, I saw two things that were kind of occupying the same space. It was one of these wonderful technologies where it solved an obvious problem that developers have, which is how do you package up an application so it can run pretty much anywhere with all of its dependencies. That's fantastic. But you could also peer through it and you could see Kubernetes on the other side of it, meaning an orchestration system that enables you to build more complex applications that can be run the way that cloud-native organizations like Google ran their applications.

3:21And it did double duty. And when I saw MCP, I had that same kind of moment where you could see this technology occupying two spaces at the same time, which is very rare and it's very wonderful. And the one part of it was it hints at what the future of a interior AI native application might look like, where you start to think of the LLM as the presentation layer and view model for an application. It's starting to describe how you would formalize interfaces, what that middle tier of the application looks like, if you think about existing databases and systems being the sort of persistence tier for these modern applications.

3:58But it also pointed to a set of capabilities that I think are going to be extremely exciting to, not just exciting, extremely necessary to large organizations. Because as you're bringing agentic systems into your environment, they are stochastic. There's new rules. They can be quite difficult to integrate sustainably and securely. And MCP really represents this small, sharp protocol that you can use to start reconciling the behavior of systems that are accessing the real world and also setting up guardrails and controls. And so I saw this and I was really captivated by it. And it started causing me to ask questions like, hey, I think I can see what the future might look like.

4:41And I think that we need to be able to operationalize this layer for organizations. And that really motivated me to start working on the work that we've been doing here at StackWell. And just so people, I mean, we've talked about MCP on the show before. Some audience members may remember that some, maybe this is their first episode. Could you just give us a kind of high level of maybe a little bit of the why of MCP and then the what of what it is? Yeah. So, I mean, the why of MCP, like you have this new technology called a transformer, right? Like a large language model, generative AI. And one of the most remarkable and wonderful things about it is that it interacts with you in natural language.

5:23It's very good at interacting with your natural language. is very good at dragging semantic meaning out of large quantities of information. And it's about taking data, turning data into knowledge, taking knowledge, turning knowledge into a decision, and then turning a decision into an action. It's this new capability. But the trick here is that there are some things that's good and there's some things that's bad at. It is really programmed to work in a kind of natural language format. So asking it to interact with relatively traditional APIs can be very fiddly. It's not necessarily set up to deal with the authentication authorization process.

6:00You know, it sometimes isn't necessarily to be completely trusted. And so by introducing model context protocol, effectively what Anthropic did was start to describe the outside world in relatively simplistic natural language terms with a JSON schema backing it that would enable large language models to start reasoning about what tools exist and being able to start invoking those tools in a deterministic way. And so the way I think about it is effectively the selectively permeable membrane that an organization can wrap around its existing systems that allows value to flow through in both directions, but enables you to start asserting the controls that you need to be able to enable AI systems to actually do real work in the real world.

6:38Yeah, that's amazing. And could you give maybe just some concrete examples of some of those kinds of systems within an organization that you might want to tie in this way? I know there's like innumerable and people could imagine anything, but just so that people have some concrete examples. Yeah, I mean, so let's just imagine the workflow of a recruiter, right? Like a recruiter is going to be doing their work on a day-to-day basis and they're going to be interacting with email. They're going to be interacting with LinkedIn. They're going to be interacting with the CMS system. They're going to be interacting with the calendaring system, et cetera, right?

7:14Now, the way that they would do their work today is they would do a lot of jumping between different SaaS systems. So I might go to Gmail and send a note to a candidate. I might go to the Google Calendar app to calendar something. Or I might have my content, my candidate management system that has some natural integration to support these things. But I'm constantly jumping around. And what I'd ideally like to be able to do is have an AI system that is able to access my email, is able to access my calendar, is able to access my candidate management system, but is able to do that in a way that I have some level of control over.

7:49I don't want this thing just going rampant and sending things. And so what MCP enables you to do is to start taking all those systems and describing nouns and verbs like, what is a candidate? What is a calendar invitation? What are the actions I want to perform? Schedule an interview. Do this type thing. And start to describe those as discrete resources that the AI model can go and acquire. and tools that the AI can actually invoke to do work in the real world. And by bubbling it up to a level where these things are described in relatively simplistic terms and then presented to the LLM, it can now discover tools that are available to it, where it's like, oh, you want to do this?

8:29Well, let me see if I can access your calendar. Oh, I can, because this calendar is now available through this MCP server. And what it also enables you to do is start dealing with the authentication authorization problem, right? that that recruiter has an identity. You know, that could be like an Okta or be an Entro or one of these IDPs. Being able to now set it up to say like, hey, an agent that's working on behalf of this recruiter should be able to access these systems as the recruiter is able to, but also with certain levels of controls in place because you don't necessarily just want to give these things unfettered access.

9:03And so MCP is really the gateway to value for enterprise systems. And this is why I think people are so excited And yeah, just as to, I guess, add to that, even literally earlier today, like three hours ago, I was sitting at my desk in our open office and I heard one of our go-to-market folks go over to one of our engineers and say, hey, I need to set up these, I forget how many it was, four or five MCP servers in the agent platform that we use in our platform. and the engineer would say like, why, I mean, I think his mind was in like cloud code world. It's like, I use my agent and all my MCP servers to my technical stuff, but this goes beyond that.

9:53Certainly it applies to developer tools because obviously agentic things power developers now, but it certainly goes beyond that. I think the ones he was pointing to, I think it was HubSpot and Instantly and LinkedIn or I forget which ones, but that level of things. And it's really interesting because I think, you know, what we're seeing, you know, look, let's be clear, LLMs are really great at writing code, right? Like it's one of the things that they do tremendously well, but they're getting increasingly good at a lot of other things as well. And, you know, like when I think about, you know, my own personal workflow, you know, how I tend to do work, you know, we've been partnering with Anthropic for a while.

10:31They introduced a tunnel. So we now have all of the Stacklock internal knowledge management systems are available through that tunnel to Claude. So it's not necessarily tethered to my desktop. I can use this on my phone. I can ask questions on my phone about things that are in my email. You know, it becomes a very powerful way to kind of open up the system. in a way that's sort of sustainable. And it just fundamentally redefines how people work. But I think the key thing here is that, what most people are experiencing today and they assume is that this is really tied to the developer's machine.

11:04If you look at the way that flawed code is structured, that co-work is structured, it's using the developer's desktop as the aggregation point where all of the state is coming in and all of the outbound connections are being terminated. And now what I think we're going to start to see is this need to be able to move that off the developer's desktop So you actually have this controlled entry point into an environment where an organization can think more holistically about what services you want to provide, but can also do it for Claude and OpenAI and Gemini and all of the other technologies. So it's this great democratizer.

11:35It's democratizing data access while preserving control. And it's doing it in a way that's not tied to a specific provider. So you go through the exercise of exposing your data once, setting up the policies once, and now you have the ability to use a pretty broad cross-sectional models. It occurs to me as we're talking about this, and I think for anyone that's listening or watching that has been kind of in it, this is very, very helpful. But I'm also a little bit worried about people joining. Could you kind of take a second and talk about what the stack looks like? Now you have MCP in there. You've kind of talked about some of the virtues of that and some of the problems.

12:17But could we actually take a second and look at what the whole stack looks like? Because we mentioned a whole bunch of different technologies over the last few minutes. And because I think one of the challenges I keep hearing this year is people trying to kind of keep up with all the new things that are in infrastructure. As you roll this world, we're all living in this world. but there's a lot of questions and people are hearing things. You know, we've talked about other technologies that we haven't brought in, things like a clawed claw and open claw and all these other things. And people are so confused in conversations that I'm having outside the podcast.

12:53Can you take just a little bit of a moment before we move forward into some of the specific things that you guys are addressing and just kind of talk about the stack and how does it all fit together and how people should think about that a little bit, just to level set before we dive a little bit deeper because we've already kind of gone into some pretty cool stuff and I don't want to leave people by. Yeah, no, no, I think it's, and I totally respect that, you know, like everyone's at a different point in the journey, right? Like a lot of us are geeking about like, you know, how do we wire in this specific tool and optimize tool calling and deal through pollution?

13:23And a lot of folks are being like, what is a tool and why would I use one, right? So I totally respect that there's a pretty broad cross-section. I mean, it's, you know, asking me to describe the stack, it's a very open system and there's a lot of different interpretations of what the stack is but let me let me describe you know what what problem that we might be solving and then we describe the set of technologies that might be used to solve that problem right so i think you know right now what most people are experiencing and what works really well for a lot of organizations is just go by anthropic right like like you know like it's like it's almost this world where it's like back in the days of the mainframe just go by no one got fired for for buying blue you know like big blue like you know like it's the it's the warm blue blanket that tells you.

14:02Ananthropic is definitely kind of, you know, starting to look and feel a lot like, you know, IBM did around the dawn of the PC. And for good reasons, they're doing amazing work. Like, you know, like we are, you know, rabid fans of their technology. And so for a lot of organizations, the starting point might be, hey, let me go get code code, or let me go get XYZ. And the starting point for them is like, okay, I've got this system, and, you know, it has access to my local file system. And so I can, you know, if I want to, you know, develop code, I can go grab some code and copy it locally, and then I can have it futz around with that code.

14:33Now I'll get to a point where I need more than the ability to just deliver, you know, this sort of code. I want to integrate this into, you know, a variety of different other systems. Like I want to be able to integrate this into technology like GitHub, or I want to be able to integrate this into Slack, or I want to be able to integrate this into whatever. And the starting point usually is, okay, well, you know, Claude will offer its own kind of native integration system. So they've gone and partnered with Slack and they've gone and partnered with Google and they've partnered with a variety of people.

15:05And you get these basic integrations in place and it works pretty damn well. You know, you get it, you alterize it, native integration works. But at some point you're going to start asking questions about like, well, what about all the other systems, you know, that I use to do my work? How do I start to expose those? And so I raised questions around like, what do I need to build a bridge between this AI system that I use to do my work and this big wide world of other technologies out there? And so for a lot of organizations, the answer is I need an M2P platform. And I think there's kind of what I think of as being four pieces that go into that.

15:39The first piece that goes into that is I need a runtime. I need somewhere where I can run this thing so that it's hosted. In the case of most developing technologies, people go NPX run. They basically just pull a package down from the internet and it runs locally. God help you if that happens to have been exploited by a hacker or something like that. And so, you know, having a secure runtime environment for this kind of makes sense. The second thing you need is a registry. Like, what servers do I want to use? Like, how do I know whether they're good or bad? You know, can I actually, you know, provide a list of servers to my organization that I might use?

16:09So the registry is the next important piece. The following piece of that is a gateway. So, hey, okay, here's the service. like I want to run them in an environment. I want to have a single endpoint that exposes them to something like Claude or to Codex or to any other systems. You need that kind of gateway technology. And then the final piece is what I think was a control plane. As you go from one to 10 to 100 to 1 ,000 servers, as you're starting to reason about mapping servers to specific user groups, you need to be able to do that. And so that kind of starts building up this kind of what we think of as the sort of MCP gateway system, like the MCP kind of platform system.

16:46But that's not necessarily all that people need. The other thing that people start to look at is also, well, what about the LLM gateway? Maybe I want to start building my own agents. Maybe I want to start reasoning about using a variety of different models. Maybe I want to be able to institute my own tracking and policy management around who can talk to what. So an LLM gateway is another compliment. I think of those as being kind of two bookends to any kind of agentic platform that a real world organization will want to use. You need an LLM gateway so you can start to direct traffic to a variety of different models and assert controls.

17:20And then you need this kind of MCP gateway so that you can start to connect real world systems. And then between those two bookends, it gets really fun and interesting. We can talk a lot about harnesses. We can talk a lot about memory management systems, session management. There's a lot of moving boxes. We can talk about agentic frameworks like N8N and how they fit into that or, you know, or CRU or LandGraph or, you know, it gets more and more and more detailed. But what I tend to think about is, and the guidance I tend to give most enterprises is like, look, you know, start with a vertically integrated system and then see how far you can get.

17:53Then start to assert, you know, assess an appetite and ask questions like, as for our developers on cloud code, so for our knowledge workers, what does it look like to get there? And you will inevitably, you know, realize that you really do need these two integration bookends. You need an MCP gateway and you need an LLM gateway. Those two things, you know, typically you want to kind of deploy together. And then there's going to be a lot of other constituent pieces that you might pull into that to start creating really great experiences for your knowledge workers that kind of decouple you from your vertically integrated AI platforms.

18:21I don't know if that's helpful, but that's just how I think about the space. No, it is a great, really good explanation there. I appreciate that. if you've been listening to the show over the past few months you realize just how transformative agentic ai is whether that's claude code or hermes agent or custom built software that you're deploying for operational efficiencies or as new products to your customers regardless of your maturity now this is the world that we're headed towards this agentic ai world and there's a lot of security and governance teams that aren't letting these agents go into production because of risks related to agency and autonomy and how do you take care of things like prompt injections or insecure tool usage.

19:11There's a lot to take care of and that's why I'm personally spending my time outside of the show working with an amazing team of AI engineers to build Prediction Guard. Prediction Guard is an AI control plane that you run in your own infrastructure behind your firewall. Developers can build on top of this control plane using everything that they want to use. OpenAI and Anthropic compatible APIs, MCP servers, frameworks like Langchain. But all of this is plugged into a built-in governance harness that enforces your organization's AI policies. And all of that telemetry goes back to your monitoring and alerting systems.

19:50I would encourage you to check out what we're doing at predictionguard.com slash practical AI. You can schedule a demo with me and the team, and I'd love to get your feedback on what we're doing. So visit us at predictionguard.com slash practical AI. That's predictionguard.com slash practical AI. Well, Craig, I have a bunch of, I don't know how many I'll get to fit in, but I have a bunch of selfish questions just as a practical developer of some of these things. I think one of the things that is some, or maybe you could help people understand, is you have this, let's say it's an MCP server for, let's say, Salesforce or HubSpot or whatever.

20:39That is running somewhere in a runtime. Like you said, it's hosted somewhere. Then there's this like identity authentication piece that I think is often very confusing for people. Or maybe a lot of times if they're building their own MCP server, they say, oh, well, here's this API to X system. And I have an API key for that API. So I'll set that as an environment variable and just all of my traffic will go to that API. but then you lose that identity piece for who who's using that do they have access to the data they should or shouldn't have access to could you help us understand like how that piece fits in the identity of the user the uh authentication with the mcp server what are the kind of best practices around that and some of the things that people could think about yeah i mean i think there's there's uh there's a lot to unpack here right and i think you know there's the the world that is and And there's the world that we hope to move into together.

21:39You know, I think this is probably, you know, it's funny, my buddy Joe, who I've worked with for years, you know, we built Compute Engine together and Kubernetes and Heptio and Tanzu. And like now he's my CTO here at Stacklock. He wrote the Spiffy paper. I don't know if you've heard of Spiffy. It's an identity system that's kind of a sort of zero trust, you know, kind of identity framework. And he wrote that paper about 10 years ago. So I think we're finally now at a point where AI is the thing that's going to kick us over the line to actually move past relatively traditional kind of OIDC-based systems to something like that.

22:13But let me kind of tear this apart into pieces. So first and foremost, MCP as a specification was really grounded in OAuth 2 workflows. So the idea being that the way that Anthropik certainly looked at the world is you have a user, they're using Claude, they have an OIDC token, that token can then get pushed onto an MCP service. It's basically identifying the user to the server. And then what happens on the back end of that is broadly an exercise to the reader. So basically, whatever you want to do. And I think there's really two problems that have to be answered. The first is an authentication problem, and the second is an authorization problem.

22:52And obviously, the sort of resources that you're accessing are going to be, you know, sort of varied. The only thing that we really have right now that works on most organizations is the existing OIDC kind of tokens. And so I think we just have to accept that's where we are. But over time, as we're starting to build agents, agents are going to have to have their own identity. And it cannot be, you know, as simplistic as the way that we've structured identity today, because it's really going to be this kind of three-legged stool. There's what I think of as a service account identity that's effectively identifying the agentic endpoint.

23:25It's like you're speaking to this specific agent. There's a set of claims that are basically provided or presented to that endpoint based on the role that the owner of the agent is provided. And then there's a set of on behalf of claims that are going to be inherited from the user who's accessing that agent. And that could then get chained through a variety of things. So there's a lot of really interesting work being done, both in the MCP upstream specification. You can start looking at things like

23:53the transaction tokens. And there's a lot of innovation happening in the IDP space around this. But that's only going to help us tomorrow. It's not going to help us today. We actually have to get through the definition and implementation of these systems. And so for most people today, I think what tends to work is you first need to institute some kind of token exchange. So typically, you don't want to be in a situation where you receive a user credential and then you pass it on to another system. You want to make sure that you're de-scoping the claims to the minimum set of claims necessary to perform a task.

24:28And so typically what we tend to do when we work with organizations is institute some kind of token exchange. So it could be, you know, there's four or five different patterns here that might make sense. You have, you know, straight pass through, you know, where the API receives an OIDC token. You could have federated trust where you have to basically exchange the token to another federated trust domain. You have the situation you talked about where you basically have to exchange it for an API key. And you need to be able to make sure that that action is actually pulled out of the agent's purview and is handled individually.