Building the internet’s next infrastructure layer | Cloudflare's Brendan Irvine-Broque

24:55Everything that went through was different. MCP was the same because it's modular and there became an MCP for everything, right? You had this sudden proliferation once, you know, like you said earlier in 2025, when it started to catch on, you started to get these MCP server directories and a repo for every kind of MCP connector that there was. And, you know, everyone has read the articles on Medium of folks being like, you can't just map one-to-one your API to MCP, but that doesn't mean no one did. So all of the APIs that do exist, they've kind of been started to push through this MCP world, right?

25:28I think you're right, though, that it comes down to like finding like the value and the usage. And you saw it as like, oh, wow, there's something to experiment with here. There's like a building block that people can use. And that's what Cloudflare, I think, is looking for. Like with things like durable objects and the whole Cloudflare ecosystem is about the building blocks to deliver the web. So MCP was evolving as one of those. I mean, at the end of the day, what people are trying to do, right, is that they're trying to figure out as businesses, how do they fit into all of these different kind of AI clients, whether it's ChatGPT or Claude or Cursor or whatever is going to come next.

26:06And recognizing that what you build when that is the interface that your customers use looks a lot different than a website with its own interface. You know, in some sense, I mean, this is now something that I feel like everybody says, but at the time we were saying it ourselves was like, these different AI tools are like the new web browsers. You know, you go first to chat GPT, you don't go to Chrome. And I think at the time, the only thing we believed in strongly was like, I don't know, the future probably isn't just more and more and more Chrome tabs. I think we all hit a breaking point in our browser around that and just trying to help internet businesses and companies figure out how they plug in.

26:44And the auth piece of MCP, especially initially, was really the hardest hurdle for people to get through if we wanted to make that leap early on from this is all local and really only for all of us software engineers and developers to run ourselves and make something that was truly accessible that, you know, everyone in a large organization or consumers can use. Right. And accessible and secure, something that was built more to scale. And it sounds like that was the like a signal, right? Like a milestone to MCP that you had to, Cloudflare had to meet that mission first of making it secure, making it something that could work remotely and in a server, right?

27:24That multiple folks connect to a managed service has a whole different world to it than the MCP servers that live locally, right? So was that, I guess, the movement within CloudFord? You are able to do that. And then you can say, okay, maybe this has legs. Now, how do we scale this to the rest of the internet. Yeah, I mean, one of the things that we picked up on initially was, especially in the first iteration of the MCP spec, it was actually pretty hard to figure out how to manage this kind of long-lived connection that a client has to have with an MCP server. And one of the unique things about Cloudflare's developer platform is that, you know, we have this interesting primitive, we call durable objects, which you can kind of think of as these like stateful workers that they're these singletons.

28:15And so you can, you know, create an infinite number of these and you can connect to them. And we saw this like easy kind of way to say, okay, well, maybe elsewhere, it would be hard for each session with an MCP server to kind of have its own instance that it's connecting to. But there was just this like very natural fit for us. And one of the things that I'm really excited about right now is that a lot more people are starting to recognize that MCP isn't just this protocol for like, you know, an agent or some other tool to talk to a set of APIs, but can be a protocol for agents talking to other agents.

28:53And the reason that that was kind of this thing that hit us over the head was, well, our model has always been with the things that we build with agents that they're all, you know, you can create an infinite number of instances of agents. And it's just a kind of like naturally fit, fit like a glove. And so I'm really excited about that future of like MCP as a protocol for agents to communicate. Yeah, I completely agree with you. I'm familiar with durable objects. I've used them before. And that's the exact mapping that I see too is like, wow, talk about how Cloudflare is already built to deliver that exactly how it was.

29:28And so it was just really cool to see that internal journey and how you'll meet the mission. And when you take that focus outward, you know, You yourself are in this really unique position where you take all of this product knowledge and this really global domain knowledge from Cloudflare and you bring these to these engineering leaders who meet with you. And when you do sit down with them, you see a lot of patterns emerging. That's what you and I talked about in our initial conversation. Patterns between all of these leaders, the questions they ask, the concerns that they have. So what do you think is like the number one question burning in all of their heads right now?

30:03So many of the engineering leaders I talked to have done a wonderful job of kind of spinning up these tiger teams or small groups of engineers who are, whether they're creating agents or MCP servers or different ways that their company can use AI. and what they're trying to do is they're trying to say how am I going to give this really big org where I you know don't know everybody and maybe there's a new hire who's going to show up next Tuesday and how do I do that safely and how do I also just understand what people are using or getting value out of or kind of roll this out securely we heard that over and over and over that just like such an interesting consistent theme is like you know I think engineering leaders really get it, that this is what's going to accelerate their organization.

30:46And so we spent a bunch of time kind of building these kind of MCP portals back into Cloudflare's set of zero trust products to be able to give people these tools to kind of administer a set of MTP servers to their team and kind of create this gateway that they can hand over and say, here's the set of tools that, maybe me as an engineering director, I want this set of teams to be able to use. And maybe there's a different set of tools that a different group of users, I want to make sure that they have access to. So that kind of rollout and management of everything is a big problem in this space.

31:21And so I'm excited about the things that we're trying to do for people there. Yeah, the rollout is a challenge because part of it is the technology. Another big part of it is like the people and the communication process behind it. I like that you called out that everyone's done a really good job at Tiger Teams. I think that is a universal cost of where everyone has identified the opportunity. fostered that curiosity, hopefully at this point, and have assembled those first movers, those heavy users, those early adopters, and gotten them to kind of start pulling the cart forward, right? And, you know, our own company, we have those internally.

31:55Many of our listeners, they relate with the story too. But outside of that, like, you know, you get that cluster of people who are very much aboard and can see it. You know, getting more people on board can be more difficult, especially when you're dealing with folks with all different levels of AI interaction and comfortableness, you know? So do you see teams get stuck anywhere or are there things that jump out at you as opportunities? I mean, one of the most basic ways that people get stuck is because in some sense, progress has happened so much faster than I think many people would have expected with so many MCP servers getting built.

32:31And a lot of these servers expose so many different tools. And it's really easy to say, well, you know, I need access to this, this, this, this, and this, and install a bunch of things. And suddenly you've blown up the context window. LLM is quite confused because you've given it 25 tools that all start with the word update. And, you know, somebody comes along and maybe they're less familiar with context engineering and, you know, how this stuff works behind the scenes. And they're just like, it doesn't really work for me. I'm not getting the results that I want. And so, you know, a lot of what I see people going through right now who are building MCP servers is starting to rethink, like, what tools should or shouldn't be available.

33:10Maybe we've gone overkill. Maybe we've exposed too many things. Or more nuance in terms of, like, which tools I actually do want to enable for certain use cases in different contexts or groups of people. You know, I think that everybody, every team wants to move really fast with AI. And a lot of people made these big pushes to get MCP servers out the door. And now we're just in this cycle of people saying like, well, how do you measure the success of an MCP tool? And you think about that, it's actually a really, really hard problem of like, oh, yeah, the tool runs. Does it give you a response?

33:44Like, yes, it's available. But like, is that response good? Well, I can pull on this thread and say, well, I can do my own evals for it. But I'm dreaming up how I use it. You might use it in a totally different way. Like evals for tools are remarkably hard, especially when you don't control the client. Yeah, you've called out a really good thing here. We talked about it a bit on Dev Interrupted as well about evals in the world of understanding how AI is used and how, you know, they're a helpful tool, but they're just one part of the picture. And there's other things too, like more of an observability level of understanding even what was the data that went into it, making that ultimate response that it did.

34:23because in many cases, you have MCP servers that are maybe connecting to data platforms and surfacing information. And they're doing that with tool calls, sure. But then like, you know, what data did it actually get? Where did that data come from? In places with a lot of data, that is a huge question. So there's all these different parts that have to be like very carefully utilized. And I also like that you called out that if you have a zillion tools, like it's not going to work. I have experienced this. If you are listening to this and you are using MCP servers and you have a whole bunch of them hooked up to something right now, like cursor or VS Code, I've been begging you, please go in there and turn some of them off or understanding, understand how many tools that you are exposing to your LLM at any given time, because it really does up the level on confusion.

35:06And so like all of these little pitfalls, right, we're learning them all together as an industry in real time. We're all writing the blog posts about them. We're all rediscovering those engineering principles underneath. And many leaders are building stuff on top of this, too. And it sounds like maybe there's a framework that folks should bring to it when making decisions around MCP adoption. And maybe those are things that are rooted in like security or compliance. But I'm just wondering if there's any like big themes that jump out at you. One that I've seen once kind of category of MCP servers I've seen work remarkably well has been everything around observability.

35:45And, you know, one of the classic kind of concerns that engineering leaders I've had when I've kind to run engineering teams before is you maybe have four or five different tools and systems for how you get logs, traces, metrics, data, you know, just like a data dog, a honeycomb, a cloudflare, all these things. Right. And how do you pull this stuff all together? You know, you go to debug things in an incident and you're saying like, well, I got to pull up six or seven different tabs in Chrome and like correlate different things just to figure out like, okay, it's this. and you know i've talked to so many people who have found success just saying here's an mcp server where i can query kind of logs or metrics or data from one system and i can bring in the other mcp server and i can have an agent just look at correlation and look at what might be happening here and piece the two things together and the reason i bring it up is that it's just a successful use case, but it's also pretty universal to engineering teams.

36:48Everybody has to debug things and it's reasonably focused and that like, you know, you're not trying to roll out all of the possible things that people could do with tools and AI at once. And it's fundamentally, you know, it's read only. And so some of the things that we're worried about of, you know, exposing a tool that can then mutate and change data or do some dangerous operation, you know, somewhat start to fade. And, you know, if you've done the right sanitization of logs for PII and other things, like you're actually in a pretty good, good spot. And so that's the, that's the use case that I'm just seeing like work pretty consistently out of the box.

37:25Yeah. And it's taken us time, obviously, as an industry to even get there, right? Because MCP has been on the scene now for about a year or so, like we're coming up on it being in the scene for a little while, but it's gone through a lot of phases of evolution in that time and adoption and becoming more mainstream. And, you know, something that you mentioned in our initial call is a lot of folks that were early to the MCP scene and building stuff on top of it and making tools with it, then would have to maybe like reset or kind of go back to the drawing board once they tried to take it out to more broadly their teams or however they wanted to ship it.

37:58So what was like the most common, like, you know, we got a wrong moment that you saw from your perspective in that world? I mean, you know, the classic thing of translate an API schema one-to-one into a bunch of tools. I mean, that's the clearest one. I think, you know, there's interesting nuances to this, right? Where it's like forgetting about the importance of descriptions. You know, at the end of the day, we're also used to, because we're engineers, we're programmers, being like, okay, well, the method is called this. And so, like, yeah, yeah, I need to leave, like, a js.comment that describes stuff.

38:33but like half the time I don't really remember to do it that well. Like to the model, that's like the information that tells you what you're supposed to use this for and gives you the context. It's literally like it's in the context window. And we, you know, some of the, some of the patterns that I've seen people try and work pretty well is like, I've seen people develop some interesting mini evals frameworks to say like, how would I iterate on my tool descriptions to figure out what's actually going to get me the best results. And just really thinking about the fact that I'm not, I'm not taking a set of APIs and exposing them just as like APIs, the way that my programmer brain would work.

39:14But like, I have to think about there's a model at the end of the day on the other side of this. And what is it? What is it doing with all of this? Yeah, it's almost like there was a there was a call to action where people learned in there that it was about combining those API calls together into what made the most sense. It was about, So, you know, in the old world of traditional engineering, where you would maybe have that function that updates something in a person's cart, maybe it fetches the cart in the first place, even fetches info about the user, maybe it updates, maybe has to fetch the product list and then fill that product.

39:47Maybe there's a whole bunch of API calls in there. If you map that one-to-one, we all learned that now you're asking the LLM to successfully thread four or five tool call interactions together. Maybe, like you said, you have 10 tools enabled and they all have an update function in there somewhere. So it's trying to pick even the right tool in the toolbox to pick up. So kind of dialing up the determinism is really important. And that's what MCP gives us with AI. I think that's why it's a powerful primitive. And then additionally, and this is kind of where I'm curious to lean on your perspective, your expertise here at Cloudflare is, you know, you bring it into the infrastructure, you make it remote, you put it behind authentication, you put it somewhere where it's a managed service.

40:31And now suddenly you can see everything it talks to, everything that goes in and out of it, all of the tool calls, all of everything that it learns and understands. And what does that level of observability like give you that people who are maybe only up until now have experienced local MCP servers, they don't even know about it. That's interesting. I guess there's two layers to this. One is, what is the inference that's happening? You know, at Cloudflare, we have a product called AI Gateway that lets kind of sits in between an application and kind of upstream inference, whether that's running on Cloudflare, or that's running on, you know, OpenAI, Anthropic, et cetera.

41:11So it gives you like one layer and there's other platforms that do this as well of like, what are the inference calls that are happening? And then there's like, well, what are the tool calls that are happening? And you kind of need both to paint the full picture of what happened and to understand like, you know, kind of like almost session replay what's going on. and I think that once you start to look at that and see real world use cases the thing that I think we're seeing most now is how many times we're going back through the agentic loop how many times we're going and saying like run inference and call tool and come back to the model the model's going to think for a second go call a tool and we're just going to go over and over.

41:57And that's great because, you know, these models are phenomenal and amazing, but they're also, you know, takes time, meaningful cost. And there's a lot of really, really interesting people, I think right now, this summer, who are kind of pushing at the edges of what might be different compute models here, like their agents that write their own code. There's all kinds of approaches to this that may make this just way, way, way, way faster or way, way less expensive to run. Yeah. So, you know, remote MCP being something that I think people are still getting used to and learning to adopt, right? And there's a lot of like questions that it solves around taking something like this to production or even letting folks like your customers access it, maybe in a read-only way to use your data as a primitive for their own AI workloads and tools, all things that we're experimenting with and that matter right now to engineering teams that are shipping products used by engineers, especially.

42:56So like, are there maybe less obvious, there's obvious security advantages to making it a managed service, but are there less obvious ones that like at Cloudflare that y 'all have become more aware of as people pull it out of the local computer grasp and put it somewhere remotely? I mean, one that I think is a little bit underappreciated is almost everybody who's built one of these things to date, what they're doing at the end of the day is they're taking an existing API that they walked in the door with that already existed coming in 2025. And they're putting in some MCP server, some layer in front that sits between a client and that API.

43:36And, you know, that's why this stuff has grown really fast is that you can do that. you can just kind of retrofit this existing piece of software. And 2025 is the year of OAuth RFCs becoming the hottest topic in the world. Absolutely. I have read more this year than any other year in my life combined. And what gets fun is you realize like, wow, there's all of these amazing specs that, you know, people have been fighting for for years and trying to make happen. But maybe there wasn't the use case to push it over the edge around like, you know, finer grain authorization, et cetera. And most of these downstream APIs that people already have don't really support as fine grain authorization as, you know, we company building them probably would want in this age of AI where this stuff really starts to matter.

44:25And I say all that because this middle layer of an MCP server, it actually sits in an interesting role where if you wanted to enforce a different kind of more granular form of authorization, operation you can do things like that or you know you're building on top of an existing api you offer and it doesn't offer the right granularity of scopes you could enforce some of that layer within an mcp server or enforce different types of guardrails and if there's a dangerous operation you can kind of hedge against that so there's some interesting threads to pull on there where you can kind of sit in the middle.

45:04The other one that we've done some cool demos around, Dina Kozlov, who's on my team, work really closely with, she did a great presentation at the MCP conference a couple months ago about MCP servers that have their own memory and can remember what a user has asked them in the past. And this is one of these questions that I think is getting asked by everybody who's investing in AI companies, right? Like, is the user's memory just going to be a thing that open AI controls or anthropic controls? And there's some great ideas, both like within our own developer platform with startups that are built on top of Cloudflare that kind of let people bring memory across platforms and persist that over time.

45:48that's really interesting it it really highlights how as a layer a remote mcp servers become almost like a decentralization layer that allows people to control that bespoke experience that they bring with them they plug into other tools that maybe remembers things from them in other sessions or how they use things or what they're authenticated for it kind of um it decouples from the llm provider from going to chat GPT or going to claw directly, you know, it decouples those things like the tool, but also memory experience, personalization. That's an interesting customization tool and also the one around security or rather enforcing scopes because I know that's a paradigm folks have been grappling with.

46:38So, you know, maybe we'll follow up with you to get a link to some of that for the show notes so folks can go check it out and especially learn more about the, talk that you mentioned as well. I'm curious too, just kind of on like a user level, you know, Brendan, are there MCP servers that you like to use that maybe you like to call out? Or I'm curious, how do MCPs fit into your daily workflows? I mean, I'm biased. I spend most of my waking hours probably here at Cloudflare. So I love working with our observability MCP server at Cloudflare. We've made a big bet there on just like trying to make observability something that's first class on our platform and is built in.

47:16So I'm always using that to kind of debug things that I'm building or trying out on our platform for the first time. I think that there's some really phenomenal examples that were built on Cloudflare by the team over at Block and Square. I remember seeing a demo when we did our demo day with them and they built this really cool thing where you could take a picture of a menu and then it analyzed the menu and it would go and take the actions and the tool calls to kind of add that to a storefront where you could order online and see the menu. And like that became that store kind of went from being in the physical world only to being online.

47:57And you think about how many steps there are in that process. But what's blown me away is that you can feed these tools, you know, higher order. Here's what I'm trying to do. Here's the project. Here's, you know, what needs to happen. And they'll do the breakdown and all the kind of like boring administrative work. And you end up in a situation where you feel very organized, even if you're not an organized person. Yeah, I really identify with that. And I like personally using the MCP tools where I can envision how they can be used with a bunch of other tools. I typically don't pick up the tool.

48:28I don't see it as like, oh, this is like a baseline primitive for working on XYZ. Like, you know, maybe something that accesses my Slack messages might be something that's useful because I can use as a launching point for a lot of different workflows, projects and things like that. So, and I love that you gave a shout out to Block. We love Block here at Dev Interrupted, especially everything that they're working on with Goose. I've done live streams with them as well with Goose and then trying out different MCP servers. So we love everything over at Block and highly agree that there's some really cool innovation happening out there.

49:00And as we've kind of talked now, you know, this whole time about the MCP world and how it's grown within Cloudflare and Cloudflare is really unique perspective on this whole problem. We've kind of dug into why remote MCP is important. But is there anything top of mind that you want to leave our audience with? Because I think that there's going to be new revelations on the future about how this tool is really used and how things like MCP can deliver even like UI experiences into chat. There's this new thing. It's relatively new, at least, MCPUI. I'm not sure if you've heard of it, with being able to basically render iframes, basically HTML fragments, almost like a shadow DOM within a chat platform.

49:43And so the idea of bringing the experiences of tools and other platforms, not just accessing the APIs through MCP, but actually bringing the experiences into the AI chats we're all using, that seems like a really cool future. I'm curious if you've seen that and what that looks like to you. Yeah, shout out to Ido and the team working on MCPUI. I remember when this was just like this early idea that a couple of people were talking about, and that feels like yesterday. I know, yesterday. I'm blown away by how quickly people have been able to make this happen. I mean, the mental model that I get excited about is that, you know, if we really take seriously this premise that your AI agent is kind of a primary interface, not that like the web browser is going away or anything like that, but like so many people are turning to that first.

50:33What, you know, what you might want to expose as a business back to those tools, you want to have some degree of influence over how that's presented back to a person. And at the start, when there was no kind of concept of UI that a MCP server can return back, I felt like, oh, this is all kind of up to the agent to decide how this all gets rendered. You know, if I give you geolocation, will you render a map correctly? And what's exciting is the idea that there could be a future, at least to me, where you You can go to these tools and get the benefits that come with that, but the services that you're talking to can still have some influence over the shape and the style and the presentation of things because they're probably the best position to say how that information, that content should be presented.

51:24But then you create these opportunities that I think a lot of people have wanted on the web for a really long time around being able to standardize interface and design units. design systems, right? Like why is it that iOS apps are often really nice to use is that there are these kind of UI patterns that are fairly hard to implement. Like UI engineering is hard, but they're part of a system and you can kind of benefit from them for free. And I do wonder a lot about if MCPUI is like a way into that for the web in a way that many people have tried in the past, web components and all the different kind of shots on goal that we've had.

52:06So I've had some great chats with Ito, Michael Greenwich, who's wonderful at WorkOS, who worked on Radix about this and excited for the future. Yeah, we are too. And excited to keep chatting about things like this. And it's been really fun, Brendan, to have you on the show and to get your perspective. Really appreciate you spending some time with us and lending your perspective to our audience. And just before we wrap, is there anywhere that, obviously everyone knows what Cloudflare is, but is there anything in particular you want to direct folks' attention to that's maybe top of mind? Yeah, go check out agents.cloudflare.com.

52:39We're doing a ton in this space of helping people build AI agents. Check out the Cloudflare blog. We've got a lot of interesting stuff coming. Love to, you know, reach out, meet different people, use Cloudflare, send me things that are good and bad. We always love to hear the feedback from everybody out there. We take our kind of role in the ecosystem pretty seriously. You can reach out to me at Irvine Broke. That's my last name. And yeah, thank you. Amazing. We'll make sure we include all of the links, the stuff that we talked about, like these blog posts and the talks and stuff and the show notes for today.

53:11Thanks for joining us here on Dev Interrupted. Reach out to us on LinkedIn. Continue the conversation. We want to hear from you all about what you thought about today's conversation. What are you doing with MCP servers? And until next time, we'll see you here on Dev Interrupted.