Model Context Protocol Deep Dive

0:03Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at Fly.io. Launch your AI apps in five minutes or less. Learn how at Fly.io.

0:44Welcome to a fully connected episode of the Practical AI podcast. In these episodes where it's just Chris and I without a guest, we try to keep you up to date with a lot of different things that are happening in the AI industry and maybe share some tips and tricks or resources that will help you level up your machine learning and AI game. I'm Daniel Whitenack. I am CEO of Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? I'm doing really good today, Daniel. We got some interesting stuff to talk about.

1:23Yeah, yeah. You know, it's been, as always, an interesting season. You know, new model releases, new tooling, new frameworks. Of course, it does seem like 2025 is set to be the year of agentic AI. And it's what a lot of people are talking about. And, you know, of course, it keeps coming up for us. Is agentic AI impacting your world in any way, shape or form? Without giving away any of the stuff I'm not allowed to talk about. Yes, it is. most definitely okay okay so yeah i would say in a lot of ways i see a bit of a pattern developing with our customers where it's kind of like they've done the rag thing so like a knowledge-based chat bot they've done maybe a structured you know data interaction maybe like text to sql something like that maybe they've created an automation like, hey, I'm going to drop this file in here and these few things will happen, some of which are driven by an LLM and then something will pop out the other end or I'll email this person.

2:47So they kind of start developing those individual assistants and then I see them start to kind of have this light bulb moment of the layer on top of those individual assistants or tools, which I think we could generally call that agentic layer, which is now saying, well, I can interact with unstructured data. I can interact with structured data. I can interact with these automations. Maybe I can interact with other systems via API. How do I start tying those things together in interesting workflows and in various ways. That's sort of what I'm seeing. I don't know. I don't know if you've seen that pattern as well.

3:30I have. And I just want to point out that you and I have been talking about like all of these things coming together for a while. We weren't before, before Agentec came out, we weren't saying it was, we weren't using the term because that's the term that ended up being, but we kind of went through, you know, a lot of the generative thing and we were kind of saying, okay, the next step is for a lot of these different architectures to tie back in. And we're definitely seeing that now. And it has a name. Yeah, and it has a name. And actually, it has a protocol. It has a protocol. Nice segue. Or anyway, it has a developing protocol, which is definitely something I think we want to dig in today.

4:12Now that we don't have a guest, we can take a step back a minute and just dig in a little bit to model context protocol. So where did you first see this pop up, Chris? Well, when Anthropic did the blog post, you know, and it started, you know, you started seeing it all over the place pretty quickly. So it was only hours after the blog post, and I'm sure it was the same for you. And then all of the follow up, you know, posts and articles have come out about it and everything, but that's, yeah, it's, it made a splash. Yeah. So technically this was last, if I'm looking at the announcement date, right, this was last year, November 25th of 2024 Anthropic released a, an announcement introducing model context protocol.

5:06Um, and of course they wrote a blog post about it. It came from Anthropic, but then also linked to an open project, Model Context Protocol, which is just Model Context Protocol at GitHub. There's a website, modelcontextprotocol.io, which kind of talks about the specification and all of that. So there is this kind of origin with Anthropic. But I think from the beginning, anthropic hope that this would be kind of more of a wide, a widely adopted protocol. And maybe we should talk about kind of the, the need for this first. We've talked a little bit about tool calling on the show before Chris. Um, I don't know if, uh, I don't know if you remember some of those discussions, but, um, there's very often this, I think we even talked about this cringe moment of people talking about an AI model, you know, renting you a car or something like that and interacting with kayak.com.

6:10Well, the AI model does not do this, right? Something else happens below the hood of that, which I think has been for some time maybe called tool calling or function calling, which is essentially the steps would be the LLM. You would give the LLM context for maybe the schema of the kayak.com API. You would then ask a query and have the LLM generate the appropriate maybe JSON body to call the kayak.com API. And then you would just use regular good old fashioned code, you know, Python requests or what have you to actually execute the API call to the external tool and get a response. Does that flow generally?

7:03You know, am I misspeaking anything? No, no, that's my understanding. There's a fair amount of kind of custom code, you know, that people would put in there to glue those different aspects together. And it varied among organizations widely. Yeah, yeah. Yeah, so that's kind of the, I guess, the stress point or the need that came about is that everyone saw, okay, maybe it's better if we plug in AI models to external tools rather than having the AI model do everything, right? So there's certain things that AI models don't do very well. And when I say AI models, I'm kind of defaulting to Gen AI models.

7:48And let's just zoom in on language models, large language models. They're not going to be your best friend in terms of doing a time series forecast, for example, maybe with a lot of data. But there's tools that do that. So what if I just leverage those tools via API and I could ask my, you know, agent, quote unquote, to give me a time series forecast of X. And there's a tool under the hood that interacts with the database, pulls the data. Then there's a tool that makes the time series forecast and boom, you kind of tie these two together. And it looks like your AI system is doing the forecast, but really you're calling these external tools under the hood.

8:31The thing is, everybody has different tools. Everybody has different databases. Everybody has different APIs. Every product has different APIs. Everybody has different function code. And so similar, I think, to in the early days of the web, everybody was sort of posting things and creating their own directories of content on the web, their own formats of things. There was no protocol necessarily that everyone followed in terms of their usage of the Internet or the web. But then there were protocols that were developed, right, like HTTP. and now it's common practice like I have a web browser right and I can go to any website and I expect certain things to be served back to me from the web server that gives me the website and those things should be in specific formats for my browser to interpret them and me to get the information and so there's a protocol or a standard in place for each of those web servers So when I visit Netflix or whatever, it's sending back similarly, let's say, structured or formatted or configured data to me as if I go to Amazon.com and I'm searching for products.

9:54This was not the case until recently with these tool calls. So an AI model, everybody was on their own to integrate tools into their AI models using whatever custom code, whatever custom prompts, whatever custom stuff, which means, Chris, if you have created a tool calling agent, and I now want to use some of your tools that you've developed, and I have my own tool calling agent, I may have to modify my agent to use your tools, or you may have to modify your tools to be compatible with my agentic framework. And that's kind of the situation that we've been in for some time, which I guess is painful, reasonably painful.

10:44It is. It's, you know, we've, we've talked about this general idea a number of times across a number of episodes. And it kind of, in my mind, it kind of comes back to that notion of the AI field maturing over time. And we've talked a lot about the fact that AI is not a standalone thing. It's in a software ecosystem of various capabilities, many of which become standardized. And I think this is yet another step of this field that we have been in and that is raging forward, maturing naturally in the way that it needs to go. Now we have a protocol that operates as that standardized glue that can be adopted and everyone knows what to expect.

11:29Just a great analogy with HTTP, as you pointed out, in terms of being, you know, you have a standard format, standard serialization, and you can plug right into it. So yeah, this was good news from my standpoint. Yeah, and I think I had a ton of questions about this because it certainly impacts the world that I'm working in directly. Indeed. And it brings up all sorts of questions. So, you know, questions like, well, how do you build an MCP server? Who's creating MCP servers? What kind of model can interact with an MCP server? What, you know, payloads go back and forth? And so it may be worth just kind of digging into a couple of those things.

12:13So first off, it may be good to dig in a little bit to what kind of how an MCP system operates. And then that may make sense. Some of the other things that we talk about in terms of models that can interact with MCP servers. So there's a series of great posts, which we'll link in the show notes. I encourage all of you to, you know, take a look at those mini posts. We'll, of course, post the main blog post for Anthropic, but also the protocol website, but a few of these blog posts as well. So one of the ones that I found that I really liked was from Phil Schmidt, Model Context Protocol and Overview.

12:52And this one is really useful. And I think it's useful because it helps you form a mental model for the various components and how they interact in one of these systems. So, you know, you might be in your car listening to this. I'll kind of talk through these main components because you might not be looking at the article. But in the system that's using MCP, there are hosts, there are clients, and then there are servers. So the host would be an application that, like the end user application. So let's say this is my code editor. And my code editor under the hood somehow is going to use MCP stuff.

13:33I'm going to be coding. I'm going to be vibe coding. right? And, you know, ask for various things and it's going to interact, you know, in the, in the IDE somehow and cause things to happen. So that's the, that's the host, the sort of end user application. There's a client which lives in, inside the host application, which is an MCP client, meaning this client might be a library, for example, that knows how to do MCP things, Like an analogy would be in Python, I can import the request package, which knows how to execute HTTP calls back and forth to web servers. Think about the client, maybe an MCP client, as a similar thing that lives within your application and knows how to do this back and forth with MCP servers instead of web servers.

14:32And then the servers are those external programs or tools or resources that you reach out to from the client. And those expose, like I say, tools, resources, prompts over this sort of standardized API. So this is, you know, it's a client server architecture. The client lives within the host, which is the end user application. And then there's the server, which you could think of, again, as this MCP server. Now, we should talk a little bit about what the MCP server does. But you could think about the client as invoking tools, making requests for resources, making requests for prompts or prompt formats or templates.

15:21The MCP server is exposing those tools, resources, and prompts. Does that make sense, Chris? It does. I mean, it is essentially it's kind of a new form of middleware in that sense, you know, for those who and I realize that term may not resonate with everybody that's listening, but that's kind of a classical term of that. The notion of connecting different aspects of services and systems together in a way to try to simplify and standardize. And so, you know, it's a different way of putting of putting it. Yeah, I think that's a great way. Even Phil in his blog post has this kind of MCP in the middle as this mediator.

16:06So I think that that's a good analogy. And, you know, we mentioned tools, resources, and prompts. So an MCP server, you know, within the specification of the protocol can expose tools, resources, or prompts. So the tools are maybe things like we already talked about with the Kayak API or calling into a database. They are functions that can perform certain actions like calling a weather API to get the current weather. Or like I mentioned, you know, booking cars or there's MCP servers that will help you perform GitHub actions right on your code base. So these are the tools or the functions that are exposed.

16:50So that's thing one that an MCP server can expose. Thing two would be resources, which you could think of as data sets or data sources that an LLM can access. So, you know, things that you want to expose either as configuration context or data sets to the application. And then the third would be prompts. And these would be kind of predefined templates that the agent can use to operate in an optimal way. So let's say that your tools in your MCP server are related to kind of question, answer, and knowledge discovery. You might have some pre-configured question and answer prompts that the LLM could use that you know would be optimized for a certain scope of work or something.

17:45You could think about it like that. So these are tools, resources, and prompts that are exposed in the MCP server. Does that make sense? It does. I know in that particular article you pointed out one of the things that I had really keyed in on that helped me kind of grok that immediately was that tools are model controlled, resources are application controlled, and prompts are user controlled. And that was easy enough for me to wrap my mind around quickly. So, yeah, that's a great explanation from you there. Yeah, yeah, definitely. So then there's kind of a couple things that are possible in the interaction between model client or MCP client and MCP server.

18:29One of the things is an application needs to understand how to connect and initialize a connection with an MCP server and sort of open that connection. that can happen over you know standard input output meaning your server might be running locally or you know as part of an application or it may be running remotely and you could interact via server sent events back and forth to the to the server but then you also need to execute a kind of discovery process i was thinking back to the good old microservices days chris which you may fondly or not fondly recall? A bit of both, depending on what I was doing.

19:19This made me think of microservices discovery things where it's like, hey, what services in my big microservices environment, how do I discover where those are at and what domain I connect to them on and those sorts of things? What are they? So this was a whole topic. I guess it maybe still is a whole topic, but there's this discovery type of mechanism where you can, in the, between the MCP server and the MCP client actually expose kind of a list of tools or a list of prompts or a list of resources. And those are discoverable to the AI application. So it knows what it can do. You know, can I book a car?

20:04No, I can't because that's not exposed as part of the MCP service, but maybe I can do GitHub related stuff or maybe I can do, you know, database related stuff or whatever that is. All right, Chris. So we've talked a little bit about MCP clients and MCP servers. There's certainly much more that is available to talk about and dig into in the protocol itself. And, you know, we've scratched a little bit of the surface here. We're not going to go through the whole protocol on the podcast. Maybe that's a relief to our listeners, but there is a whole protocol there. I think it would be good to talk, though, about kind of two additional things, which immediately popped into my mind when I saw Anthropic releasing MCP and talking about it is, number one, how do I create an MCP server or where do I get access to MCP servers to tie into my own AI system?

21:05And then secondly, well, what if I don't use Anthropic models? Can I use MCP? Those were two immediate questions from my end. And I don't know, Chris, if you've seen, there's various GitHub repos that are popping up and also examples of various MCP servers. Have you seen any that are interesting to you? The one that's most interesting to me, because when I'm not focused on AI with Python specifically, I'm very focused on Edge with Rust. And there's an official Rust SDK for the model context protocol. So that's naturally where I gravitated to. Yeah, yeah. So there's Python implementations. I think there's many programming language implementations.

22:02there's also sort of example servers that are kind of pre-built I've seen various ones for like blender which is a 3d kind of modeling animation type of thing which is open source yeah exactly and then Ableton live which is a platform that is like a music production platform there's ones for GitHub, I already mentioned that. Unity, the game development engine, there's ones that can control your browser, integration with Zapier, all sorts of things. So people have already created many, many of these MCP servers. And again, when you're creating this MCP server, basically you just have to create essentially a, you could think of it like a web server that has various routes on it, But these are specific routes that expose specific sorts of things, these tools, resources, and prompts over a certain protocol.

23:07And there is communication back, for example, back and forth of JSON, for example, over servers and events. But, you know, again, there's a specific protocol that's followed. Now you can look through all of the specific details of the protocol if you want to say create a model context protocol server for your tool. And I actually wanted to do this. So we have an internal tool that we use for doing text to SQL. It's very frequent. I often call it the kind of dashboard killer app. It's like, you know, everyone's created tons of dashboards that no one uses in their life. And wouldn't it be better if you could just connect to your database and ask natural language questions?

23:52So we have a whole API around this. And you can add your database schema information and all sorts of fun things and do natural language query. And so there's like, I don't know, six or seven different endpoints in this kind of simple little API that does structured data interaction. So I'm like, okay, cool. We've written that with fast API, which is awesome. So it's a web server. It has certain endpoints that allow you to do the SQL generation or allow you to modify database information or allow you to do various elements of this operation. And we utilize that as a tool internally via tool calling.

24:34So I thought, well, what would it take for me to convert that into an MCP server that I could plug into an agent? Well, you could kind of do that more or less from scratch, just following the protocol. But people have already started coming up with some really great tooling. So there's a thing called FastAPI MCP. So if you just search for that, this is a Python framework that essentially works with FastAPI and basically converts your FastAPI web server into an MPC server or MCP server. And it works. So, you know, from my experience, I just added a few lines of code to my fast API endpoint, wrapped my fast API application in this framework, and then ran the application, which is, again, this fast API application.

25:30And that was immediately discoverable as an MCP server, meaning that I could, if I had an AI system, which we'll talk about that bit here in a second. If I had an AI system that could interact with MCP servers, my service now, the text-to-SQL system that we use, would be available to that agent to use as a potential tool that's plugged into a database that we would connect it to. Does that make sense? It does. That was a good explanation. Yeah. So I'm sure also, I mean, you mentioned this Rust client that you talked about. I imagine a similar thing is possible there with a bunch of convenience functions and that sort of thing.

26:18I don't know Rust quite as well, but I imagine that's the case. It is. And it's one of those, I love the fact that MCP is rapidly gaining so much language support off the bat. I think you've heard me say this before. One of my pet peeves is kind of the Python only nature of a lot of AI. At least it starts there. And I think I've said in previous episodes, it's a maturity thing when you can get to where you're supporting lots of different approaches to accommodate the diversity that real life tends to throw at us. That's good. I love, and MCP has shot up that very, very quickly. So, yeah, in the world that I'm in, you know, playing it, kind of combining MCP as a protocol that works at the edge as well as in the data center is a big deal for me.

27:09Yeah, and it does actually work also kind of single node. I mean, we've talked about client server, right? But you can run an MCP quote server in this sort of embedded way that is discoverable in a desktop application or in a single node application. So I guess what I mean is, if you're using MCP, and this is security and authentication related, it doesn't mean that you need to connect over the public internet to an MCP server. And it doesn't mean that all of that is unauthenticated or you can't apply security of any type. What it does mean is that if you are, for example, in the example that I gave, so I've now converted our text to SQL engine into an MCP server, I can plug in a database connection to that, connect to a database.

28:07But depending on how I set up the connection to the database, there could be potential problematic vulnerabilities there. And if I don't have any authentication on my MCP server, you know, and I put that on on the public Internet, anyone could use it. So there's two levels of kind of security or authentication or, you know, threat protection that's relevant here. One is the actual connection level authentication to the MCP server. And the other is, well, I can still create a tool that's vulnerable, right, or has more agency than it should. Yeah. I think one of the things I love about that call out from you is that you can be operating on that one physical device and tying various systems together.

29:01And just like if you take it outside the AI world and you talk about protocols that we are commonly using, you mentioned HTTP earlier, protobufs are really common and things. You may be using all of those other ones that we've been using for years on one device. It doesn't mean that there are, by definition, many services in many different remote places. It can all be collected there. And it still brings value because you still have that standardization. And the various vendors, whether they be commercial or open source, can provide interfaces to that to make it easier. So it becomes a much more pluggable and yet not tightly integrated, which is a good thing, architecture.

29:40And I think MCP really gives us that same capability now in this space. And so it's, like I said, it really is pushing it up the maturity, you know, up the maturity level from we're all writing custom glue code to now, hey, I'm going to standardize on MCP and away we go. Yeah. And I think similar to people can carry over some of their intuitions from working with web servers into this world. like you wouldn't necessarily just download some code from github and expect there to be no vulnerabilities in it when you run that server you know locally same goes with mcp right you would you would definitely want to know what you're running you know what's included where you're running it how authentication is set up etc etc similarly if you're connecting to someone else's MCP server, like Chris, you're running one and I want to connect to it, depending on the use case that I'm working with, I may very much want to know what data does your MCP server have access to?

30:48How are you logging, caching, storing information, et cetera, et cetera? Is it multi-tenant? Is it single-tenant, et cetera, et cetera? So you can bring some of those intuitions that you have from working in the world that we all work in, which involves a lot of client server interactions and bring that into this world. Okay, Chris, we've talked about MCP in general. We've talked about creating MCP servers or the development of them. There's one kind of glaring thing here, which is Anthropic released or announced this model context protocol, and certainly others have picked up on it. And you see OpenAI also now supporting MCP where before they had this kind of their version of tool calling in the API.

31:40So there's a more general question here, which is, well, I'm using Llama 3.1 or DeepSeek. Can I use model context protocol? And more generally, as models proliferate, which they are, and people really think about being model agnostic, meaning they're building systems where they want to switch in and out models, do I have to use Anthropic or now OpenAI to use MCP? So the answer to this question, at least as far as what we've discovered in our own work, is as of now, sort of yes and no. But in the future, definitely there will be flexibility to many things. So what I mean by that is Anthropic has a kind of head start, in a sense, in the same way that OpenAI has released certain things, you know, like various agent protocols or tool calling or stuff.

32:44And they had, you know, it was something they released, right? Something they had been working towards and they had maybe an advantage initially. So Anthropic obviously has been working towards this. their models, their desktop application, et cetera, supports it well. Others are kind of playing a little bit of catch up and that would include kind of open models. So if you think about something like a Llama 3.1 or, you know, Quinn 2.5 or whatever model you're using, those open models, there's nothing preventing them from generating model context protocol aligned things. Agreed. But they haven't necessarily been trained as part of their training data set to generate those things.

33:29Meaning you can have an open model that generates what you need for model context protocol interactions, but you're probably going to have to load the prompt of that open model with many, many examples of model context protocol and information about it for it to be able to generate that, which is totally fine. You can do that. And we've done that internally and I've talked to others who have and there's blog posts about it, etc. So there's nothing in that sense. That's why I say yes and no. There's nothing preventing you from doing this with open models right now or models other than Anthropic.